Final Report 


On 


Development of Advanced Life Cycle Costing Methods for 
Technology Benefit/Cost/Risk Assessment 


Covering Tasks Performed During Year 3 
Grant Number NAG- 1-2 149 


Submitted to 

NASA Langley Research Center 
Grants and Contracting Office 
Hampton VA 23681-2199 

Technical Monitor: Mr. Robert Yackovetsky 
Mail Stop 261 

Technology Integration Office 
Hampton, VA 23681-2199 
Tel. No.: (757) 864-3844 
Fax No.: (757) 864-8093 


Report Prepared by 


Aerospace Systems Design Laboratory 
School of Aerospace Engineering 
Georgia Institute of Technology 
Atlanta, GA 30332-0150 


April 11, 2002 



Grant NAG-1-2149 Final R eport 


Objective Review 1 

Proposal Task Review 1 

Capacity Focus 2 

Progress Accomplished 4 

Task 3: Assessment of Impact of New Technologies 4 

Technology Compatibility Matrix 4 

Technology Impact Matrix 5 

Pareto Charts 7 

Prediction Profilers 7 

Probabilistic Evaluation of Technologies 10 

Dynamic Contour Environment 14 

Conclusion 16 

Task 4: Code Fidelity 18 

Objectives and Motivation 18 

Tools implemented 21 

Method of implementation 24 

Case Study Results 26 

Fidelity vs. Noise 32 

Conclusions 36 

Capacity Focus Task: Formulation of a Method to Assess Technologies for the 
Improvement of Airport Capacity 38 

Motivation 38 

Subtask 1: Capacity Model Analysis 45 

Subtask 2: Modeling the National Airspace System 47 

Subtask 3: Technology Assessment 67 

Acknowledgements 79 

References 79 


Appendix A: Pareto Charts 

Appendix B: CDF of Technology Space 

Appendix C: FPI Details 

Appendix D: RDT&E Code Fidelity Figures 


1 


Grant NAG-1-2149 Final Report 


OBJECTIVE REVIEW 

The overall objective of this three-year grant is to provide NASA Langley’s System 
Analysis Branch with improved affordability tools and methods based on probabilistic 
cost assessment techniques. In order to accomplish this objective, the Aerospace Systems 
Design Laboratory (ASDL) needs to pursue more detailed affordability, technology 
impact, and risk prediction methods and to demonstrate them on variety of advanced 
commercial transports. The affordability assessment, which is a cornerstone of ASDL 
methods, relies on the Aircraft Life Cycle Cost Analysis (ALCCA) program originally 
developed by NASA Ames Research Center and enhanced by ASDL. This grant 
proposed to improve ALCCA in support of the project objective by updating the research, 
design, test, and evaluation cost module, as well as the engine development cost module. 
Investigations into enhancements to ALCCA include improved engine development cost, 
process based costing, supportability cost, and system reliability with airline loss of 
revenue for system downtime. A probabilistic, stand-alone version of ALCCA/FLOPS 
will also be developed under this grant in order to capture the uncertainty involved in 
technology assessments. FLOPS (FLight Optimization System program) [Ref. 1] is an 
aircraft synthesis and sizing code developed by NASA Langley Research Center. This 
probabilistic version of the coupled program will be used within a Technology Impact 
Forecasting (TIP) method to determine what types of technologies would have to be 
infused in a system in order to meet customer requirements. A probabilistic analysis of 
the CER’s (cost estimating relationships) within ALCCA will also be carried out under 
this contract in order to gain some insight as to the most influential costs and the impact 
that code fidelity could have on future RDS (Robust Design Simulation) studies. 

PROPOSAL TASK REVIEW 

The tasks for this three-year program are listed below as a review of what was proposed 
in the original statement of work. A short description for each task is offered, and a 
summary of the tasks accomplished can be found in Table 1. 

Task 1: Probabilistic Cost Assessment Program 

A stand alone probabilistic ALCCA (e.g. ALCCA coupled with FPI [Ref. 2]) 
will be created as a subset of the more comprehensive FLOPS/ ALCCA/FPI 
program which will be utilized to show cost/benefit/risk tradeoffs. 

Task 2: Detailed RDT&E Costing 

A new RDT&E module with a more detailed cost breakdown, and the capability 
of accounting for the development of new technologies will be developed. 

Task 3: Assessment of Impact of New Technologies 

The infusion of new technologies for a given configuration must be considered 
when all other alternatives (optimization, opening design space, etc.) have been 
explored. However, the impact of a technology can be qualitatively assessed 
through the use of technology metric “k-factors”. These “k-factors” modify 
technical metrics, such as specific fuel consumption (SEC), lift to drag ratio 
(L/D), and component weights, that result from some analysis or sizing tool. 
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Task 4: Code Fidelity 

The development of a method to be used for evaluating the fidelity of an 
economic analysis code was proposed. The economic code chosen for this case 
study is Aircraft Life Cycle Cost Analysis (ALCCA). 

Task 5: Detailed Process Based Engine Costing 

This task proposes to develop several Response Surface Equations (RSEs) using 
COMPEAT. These RSEs will be used to link the most important cost 
parameters with their most important cost drivers. COMPEAT is an engine cost 
estimation tool developed and maintained by General Electric Aircraft Engines 
in Evendale, OH. It is a “state of the art” tool, capable of estimating program 
costs for any type of aircraft turbofan engine. Its accuracy is limited only by the 
accuracy of the database. 

Task 6: Inclusion ofTAROC and DOC+I 

This task proposes to restructure the cost calculations internal to ALCCA to 
generate the desired cost metrics for the airframe manufacturer. 

Task 7: Supportability in Cost Estimation 

This task proposes to investigate the impact of supportability issues on the 
overall economic viability of commercial aircraft. Also, the impact of 
supporting new engine technology on the overall economic viability of the 
HSCT is to be investigated. 


Table 1: Summary of Task Status 


Task No. 

Description 

Status 

1 

Probabilistic Cost Assessment Program 

Completed in Year 1. 

2 

Detailed RDT&E Costing 

Completed in Year 1. 

3 

Assessment of Impact of New Technologies 

Feasibility and Viability are assessed in 
Year 1. Technology identification and 
TIF environment are determined in 
Year 2. Technologies evaluated and 
task completed in Year 3. 

4 

Code Fidelity 

Completed in Year 3. 

5 

Detailed Process Based Engine Costing 

New Capacity Focus 

6 

Inclusion ofTAROC and DOC-\-I 

Completed in Year 1 

7 

Supportability in Cost Estimation 

New Capacity Focus. 


CAPACITY FOCUS 

Tasks 5 and 7 have been given a system-level emphasis resulting in a capacity focus task 
designed to assist NASA Ames Research Center in its efforts to accomplish the 
Throughput Technology Objectives. The new capacity focus still maintains the research 
objective of providing NASA Ames with improved affordability tools and methods by 
developing the capability to assess the economic impact of advanced aviation 
technologies. More importantly, this capacity focus task will also evaluate how these 
technologies would be used in the integrated aviation system from a probabilistic 
standpoint. The capacity task is titled “Formulation of a Method to Assess Technologies 
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for the Improvement of Airport Capacity. ” In order to accomplish this task objective, a 
strong collaboration between the Logistics Management Institute (LMI), the developers 
of Aviation System Analysis Capability [Ref. 3], has been fostered. The probabilistic 
approach to evaluate advanced aviation technology in conjunction with a process to 
understand and evaluate their impact at a system of systems level will assist NASA in 
realizing their goal of tripling the aviation system throughput, in all weather conditions, 
within 10 years, while maintaining the current level of safety. 

The capacity task is further divided into three subtasks. The first one involves the 
identification of the most influential factors when assessing capacity at an airport, 
utilizing LMI’s Capacity model, and the subsequent creation of a Technology Impact 
Forecast environment. Subtask two aims to consider the airspace system as a whole 
identifying the significant fields involved and their interactions, as well as creating an 
environment conducive to a system-of-systems technology assessment. The last subtask 
will then utilize that environment in a sample technology assessment through the use of a 
methodology such as TIES (Technology Identification, Evaluation and Selection). 


3 



Task 3: Assessment of Impact of Ne w T e chnologi es 


PROGRESS ACCOMPLISHED 

Task 3: Assessment of Impact of New Technologies 

This task proposed to investigate the effect of new technologies on a 600 passenger 
aircraft. From Year 1 tasks, the design space was investigated for technical feasibility 
and economic viability based on five performance criteria and four economic criteria. 
The design space was deemed non-feasible due to the violation of the takeoff gross 
weight limitation of one million pounds. The focus of Year 2 was to establish 
technologies that could be infused into the system and create a Technology Impact 
Forecasting (TIF) Environment based on guidance of the chosen technology impact 
factors. The focus for the current year involved evaluating those technologies and 
selecting the technology combinations with the highest potential to create a feasible 
design. 

Technology Compatibility Matrix 

A Technology Compatibility Matrix is formalized through Integrated Product Teams 
(IPTs) to establish physical compatibility rules between technologies previously 
identified as having the potential to improve system performance and/or cost, thereby 
increasing the probability of reaching project goals. The Technology Compatibility 
Matrix for the 600 passenger baseline aircraft is shown in Table 2. The purpose of the 
matrix is to establish which technologies are compatible and can thus be employed 
simultaneously. This helps drive the development of the technology space through use of 
the Technology Impact Matrix (TIM), described in the following section. By identifying 
those technologies that are not compatible, the matrix also eliminates the possibility of 
running cases with impossible technology combinations. In this matrix, a 1 indicates 
compatibility and a 0 indicates incompatibility. Therefore, any two technologies that are 
assigned a 0 will not be modeled simultaneously in any single case. For example, hybrid 
laminar flow control (HLFC) is physically incompatible with a composite wing, as the 
microholes required for HLFC would affect the structural integrity of the composite. 
Thus, this combination has been labeled with a 0. 
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Table 2: Technology Compatibility Matrix 



Composite Wing 
Composite Fuselage 
Aircraft Morphing 
Natural Laminar Flow Control 
Maneuver Load Allevaition 

AST Engine Concept 

Integrally Stiffened Aluminum 
Airframe Structure 

HLFC 

IHPTET 

T1 T2 T3 T4 T5 T6 T7 T8 T9 

Composite Wing 

T1 

111111 0 0 1 

Composite Fuselage 
Aircraft Morphing 
Natural Laminar Flow Control 
Maneuver Load Allevaition 

AST Engine Concept 

Integrally Stiffened Aluminum 
Airframe Structure 

HLFC 

IHPTET 

T2 

T3 

T4 

T5 

T6 

T7 

T8 

T9 

^ _ 


Technology Impact Matrix 

Once the Technology Compatibility Matrix is determined, the potential system and sub- 
system level impacts of each technology are established including primary benefits and 
secondary degradations. Technology Impact Forecasting ( I’lF) is a method of predicting 
the effects that future technologies will have on chosen responses, such as takeoff gross 
weight and NOx emissions. Thus, it creates an environment around the question “What 
would happen if this element of the design could be improved?” This method does not 
require information on specific technologies. Instead, it looks at the overall technological 
improvement needed in a disciplinary metric to reach a constrained target. In addition, 
this method can be used as a precursor to the determination of the impact of specific 
technologies on appropriate responses. Such predictions can become incredibly useful 
when the decision of which new technologies to invest in has to be made. 

The factors by which the disciplinary metrics or parameters are multiplied when 
technologies are added are called k-factors (or technology dials). These k-factors, while 
representative of the impacts of technology infusion, can also be used without specific 
technologies to create the Technology Impact Forecast (TIF) environment. That is, 
changing a disciplinary metric or parameter even if there is not yet a technology 
identified that can achieve the specified change. This way the effect of that disciplinary 
metric at the system level can be assessed. 

These k-factors are grouped in a vector (i.e. k_vector), since several technologies can 
influence several disciplinary metrics. Each element in this k_vector corresponds to each 
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of the different k-factors considered. Not all technologies will affect each element of the 
vector, but the vector must capture all technologies to be assessed. The vector must also 
include both benefits and penalties to accurately assess the impact of technologies on the 
objective. These vectors can then be entered into a DoE (Design of Experiments) in 
place of the original disciplinary metrics to model the impact of the newly infused 
technologies on the responses. In addition, the impact of combined technologies can be 
found by adding the k_vectors for each of the technologies being considered. This 
method assumes that the effect of the combined technologies on the disciplinary metrics 
is the sum of the effects of the individual technologies being considered. 

The Technology Impact Matrix (TIM) is a way of organizing and mapping the 
technology impacts to the k-factors that will be applied to the disciplinary metrics. The 
TIM is shown in Table 3, which lists the technologies that are being considered across the 
top of the columns. There is a disciplinary metric in each row below those headings, 
which is affected by at least one technology under consideration. Obviously, not each 
technology is going to impact each disciplinary metric, but as one reads down a column, 
the expected impact of the technology on the disciplinary metrics can be easily seen. 
Each of these columns represents the k_vector for that particular technology. 

Table 3: Technology Impact Matrix 


Technical K Factor Vector 
Wing Area 
Vertical T ail Area 
Horizontal Tail Area 
Drag 

Subsonic Fuel Flow 
Wing Weight 
Fuselage Weight 
Electrical Weight 
Engine Weight 
Hydraulics Weight 
AL Wing Stru. Man.Costs 
O&S 
RDT&E 

Production Costs 
Utilization 



-2% -2% *3% -5% 

— -0.50% -1.50% — 

-15% — -3% 

-25% -2% 


+ 2 % + 2 % 

+ 2 % + 2 % + 2 % + 2 % 

+10% +10% -3% -3% 

- 2 % - 2 % — 


I 


2 

T5 


+18% 

-40% 

-36% 

-3% 



I- 




T6 


- 10 % 


CO *1 
T7 


I 

I 

E 

5 


X Q 

T8 


- 10 % 
+ 1 % 
•15% +4% 


fc 

X 

T9 


-5% 


+5% 

+3% 


+2% 

... 


-30% 

... 

+0.5% 

-20% 

-10% 

... 

... 

... 

... 

— 


-2.50% 

... 


— 

-3% 

-2% 

+3% 

-3% 

+3% 

-4% 

... 

+4% 

+3% 

— 

-3% 

— 

+1% 

... 

— 

+3% 

+2% 

-2% 

+2% 


There are nine different technologies that are being considered, and fifteen disciplinary 
metrics that they impact. The non-dimensional k-factors in the TIM are the sum of one 
plus the percentage impacts. The maximum value of the k-factors for a given 
disciplinary metric is the sum of all of the increasing impacts, while the minimum value 
of the k-factor would be the sum of all of the decreasing impacts. To determine the 
dimensionalized k-factor impacts, the baseline values of each disciplinary metric are 
multiplied by the minimum and maximum k-factor values. 

Once the ranges for the k-factors have been selected, a DoE is generated for these twenty- 
nine k-factors. The variable values dictated by the DoE are then entered into the analysis 
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tools and new RSEs that relate the effects of the k-factors on the responses to the 
variation of the disciplinary metrics are created. Once the technologies being considered 
are mapped to the disciplinary metrics, the effects of the technologies can be found (to be 
accomplished in the next phase). For now, this mapping allows the TIF environment to 
be created. This TIF prediction profile is essentially a graphing of the partial derivatives 
of each response with respect to each technology dial. 

Pareto Charts 

Pareto charts depicting the relative contributions of the various proposed technologies on 
the desired responses were created for this task. Pareto charts enable the identification of 
the most statistically significant contributors. They are a statistical quality improvement 
tool that shows frequency, relative frequency, and cumulative frequency of a set of 
variables to a response. They are in the form of a bar chart that displays the influence of 
a variable. This allows the designer to see which technologies yield the most beneficial 
changes in a given response. 



Figure 1: Effect of Technologies on Acquisition Price 

As can be seen in the example Figure 1, Technology 4 yields the greatest benefit to the 
acquisition price by reducing it by nearly 12%. Alternately, Technology 9 has the worst 
effect on acquisition price, as the infusion of this technology causes a 2.4% increase. 
Pareto charts for the remaining responses can be seen in Appendix A. 

Prediction Profilers 

The decision-maker can also identify the technologies that most significantly impact the 
system metrics through the use of a prediction profiler. The profiler provides a dynamic 
environment through which trade-offs can be rapidly performed. A key design variable 
or technology k-factor can be altered to see instantaneously the effect on the responses. 
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A prediction profiler is shown in Figure 3 and depicts the prediction traces for each 
technology k-factor. The prediction trace is defined as the predicted response in which 
one k-factor is changed while the others are held at their current values, effectively, it 
shows the sensitivity of the response to the technology infusion. Moving the dotted line 
varies the k-factor; the underlying RSEs are reevaluated, and the prediction traces and 
response values are updated in real time. 

The prediction profilers of the technology mapping can also be interpreted as a 
forecasting environment as seen in Figure 2. If a decision-maker does not have specific 
technologies to evaluate, this mapping environment could guide the decision-maker in 
selecting appropriate technologies for infusion. This technique is called Technology 
Impact Forecasting (TIF). 

For example, since the acquisition price, TOGW, and RDT&E have very little if any 
feasible space, the decision-maker should select a set of technologies that reduce wing 
weight, engine weight, and costs. These k-factors significantly influence the metrics 
mentioned as seen by the large prediction trace slopes. Thus, once the k-factor values are 
established, the decision-maker must identify specific technologies that provide the 
desired k-factor values. 

Once those specific technologies have been identified, they can be mapped against the 
responses to see the effects of an individual technology or combinations of technologies 
on the responses of interest. Effects of the parameters in this prediction profiler are 
evaluated based on the magnitude and direction of the trace slope, where the “-1” and “1” 
values indicate whether a technology is "on" or "off." The larger the slope of the line, the 
greater the influence of a given k-factor. If a k-factor, listed on the abscissa, does not 
contribute significantly to the response listed on the ordinate, the slope is approximately 
zero. The sign of the slope, either positive or negative, depicts the direction of influence 
of the k-factor. Caution should be exercised since the compatibility rules are not inherent 
in the sensitivities, and care should be taken prior to arbitrarily turning “on” a mix of 
technologies. The prediction profiler in Figure 3 maps the nine specific technologies 
against the thirteen desired responses. As shown, this profiler depicts the effects of the 
combination of Tl, T2, T3, T4, and T5. 
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Figure 2: Technology Impact Forecasting Environment 




Task 3: Assessment of Impact of New Technologies 



Tt T2 T3 T4 T5 T6 T7 T8 T9 


Figure 3: Prediction Profile for Nine Specific Technologies 

One should not underestimate the power of the prediction profiler. Once the technology 
environment is created, the decision-maker can instantaneously quantify the impact that 
any mix of technologies has on the system under investigation without the need to re- 
execute any analysis code. Furthermore, if the anticipated impact of a technology 
changes as the development progresses, again, no analysis code execution is required. 

Probabilistic Evaluation of Technologies 

The design of complex systems is immersed in uncertainty due to incomplete knowledge 
about the system and the behavior of the system in a relevant environment. Because of 
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this uncertainty, new paradigm design methods must be probabilistic. Traditional 
methods of design space exploration were based on the designer’s intuitive knowledge of 
what the responding system might look like. A designer would perform paper study 
trades, and then build, test, fly, and modify the system as needed. This approach resulted 
in iterative designs which were both costly and time consuming. 

An alternative approach is needed that is probabilistic in nature. The motivation for a 
probabilistic evaluation is to provide a more realistic assessment of the uncertainty and 
risk associated with the impact of immature technologies. Probabilistically evaluating a 
single technology or a combination of technologies is similar to the deterministic 
evaluation, except that the k-factors are distributions rather than single point values. To 
quantify the impact on a system metric, a Monte Carlo Simulation (MCS) is performed 
with user defined frequency distributions for each k-factor element and a cumulative 
distribution function (CDF) obtained for each system metric. A Monte Carlo Simulation 
(MCS) is the most accurate probabilistic technique to simulate uncertainty, by randomly 
generating values within a pre-specified range. By linking a sophisticated analysis tool 
with MCS a cumulative distribution function (CDF) for each of the desired objectives or 
metrics, as seen in Figure 4, is produced. The CDF represents how the metric behaves as 
a result of all the possible design variable combinations and in essence, defines and 
bounds the space of interest, whether the space is design, technological, or economical in 
nature. At a probability level of 0% (P=0%), the metric value is the best that can ever be 
achieved with the defined space, assuming that the CDF’s probability levels (or P-levels) 
are increasing with increasing metric values. At P=100%, the entire space falls below the 
corresponding metric value. Any probability of achieving a solution is favorable since it 
represents the outcome of design variables. Yet, the decision-maker still strives for 
alternatives that maximize the feasible and viable design space. 


Target 



Figure 4: Generic Cumulative Distribution Function 

This process can be used to simulate the addition of new technologies to a baseline 
concept. If one assumes that the technologies are additive, then a combination of two or 
more technologies remains a simple MCS on the RSE. Now, instead of the response, R, 
being a function of only one k-vector (i.e., technology), it is a function of the sum of the 
combination of vectors (i.e., sum of technologies). For example, if one wants to 
determine a system metric value due to a combination of T1 and T2, distributions are 
assigned to each element of both technology k-vectors. Subsequently, a random number 
generator selects a value for the first element of the T1 vector and the first element from 
the T2 vector, based on the user-defined frequency distributions. Then, the two values 
are added to obtain a “new” first element that is inserted into the RSE and the system 
metric value calculated. This is done for each element and each time a new combination 
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of technologies is desired. This process is automated with the software package Crystal 
Ball®, which is a Microsoft EXCEL® “add-in” function. 

For this study, a uniform distribution was used to represent each of the fifteen k-factors, 
with the lower and upper limits established in Table 4. 

Table 4; K-factor ranges and corresponding input variables 


NAME 

NAMELIST 

VARIABLE 

LOW 

LIMIT% 

HIGH 

LIMIT% 

BASELINE 

ACTUAL 

LOW 

ACTUAL 

HIGH 

Wing area 

CONFIN 

sw 

0 

20 

8500 

8500 

10200 

Vertical tail 
area 

CONFIN 

SVT 

-40 

5 

950 

570 

997.5 

Hor. tail area 

CONHN 

SHT 

-40 

5 

1700 

1020 

1785 

Cruise drag 

MISSIN 

FCDSUB 

-30 

5 

1 

0.7 

1.05 

Subsonic 
fuel flow 

MISSIN 

FACT 

-20 

5 

1 

0.8 

1.05 

Wing weight 

WTIN 

FRWI 

-40 

5 

1 

0.6 

1.05 

Fuselage 

weight 

WTIN 

FRFU 

-30 

5 

1 

0.7 

1.05 

Elec. Weight 

WTIN 

WELEC 

-5 

10 

1 

0.95 

1.1 

Engine wt 

WTIN 

WENG 

-50 

10 

19000 

9500 

20900 

Hydraulics 

weight 

WTIN 

WHYD 

-10 

10 

1 

0.9 

1.1 

Al.wing 

manuf.Costs 

RDTE 

CFWAL 

-5 

5 

1 

0.95 

1.05 

o&s 

IWGT 

AKOANDS 

-15 

10 

0 

-0.15 

0.1 

RDT&E 

rwGT 

AKRDTE 

-10 

20 

0 

-0.1 

0.2 

Production 

costs 

IWGT 

AKPRICE 

-10 

25 

0 

-0.1 

0.25 

Utilization 

COPER 

U 

-10 

10 

5000 

4500 

5500 


The resulting cumulative distribution functions show an increase in the feasible and 
viable design space. The “new” design space with respect to the metric of takeoff gross 
weight is shown in Figure 5. Without the application of new technologies, as represented 
by the aforementioned k-factors, the baseline vehicle had a takeoff gross weight of over 
1.3 million pounds. After the improvements in the design, due to the new technologies 
are simulated, the technology space becomes viable. The baseline value for TOGW was 
dramatically improved by the application of k-factors, in fact, 90% of the available 
technology space will allow for the target value of 1,000,000 pounds to be met. The 
cumulative distribution for the remaining performance and economic metrics are included 
in Appendix B. 
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Takeoff Gross Weight 


|— ♦—Model Target Baseline j 



600,000 700.000 800,000 900,000 1,000,000 1,100,000 1,200,000 1,300,000 1,400,000 

Pounds 


Figure 5 : CDF for Takeoff Gross Weight after the application of k-factors 

Table 5 shows the changes in the probability of achieving the targets or constraints for 
each metric. 

Table 5 : Feasible and Viable Design Space 


Parameter 

Acronym 

Target or 
Constraint 

Baseline 

Feasible 

Space 

Feasible 
Space after 
k factors 

Performance 

Approach Speed 

Vapp 

<150 kts 

85% 

100% 

Landing Field Length 

LdgFL 

< 11,000 ft 

100% 

100% 

Takeoff Field Length 

TOFL 

< 11,000 ft 

5% 

100% 

Takeoff Gross Weight 

TOGW 

< 1,000,000 lbs 

0% 

88% 

Economics 

Acquisition Price 

Acq $ 

190 FY96 $M 

5% 

3% 

Research, Development, 

RDT&E 

Minimize 



Testing, and Evaluation 

Average Required Yield pier 

S/RPM 

~ $0,095 FY96 

25% 

78% 

Revenue Passenger Mile 


All of the metrics shown above, except for one, showed significant improvement in their 
ability to meet the established targets and constraints. It is apparent that the addition of 
new technologies will be beneficial to the success of this vehicle. The one metric that 
failed to improve was the acquisition price. It is important to note, as mentioned 
previously, that the “k-factors” not only include the obvious benefits but also the 
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degradations to the system as well. Often, these degradations appear in the form of 
increased investment of resources, which tends to negatively affect the acquisition price. 
Despite this effect, viable design space still exists and the target acquisition price of $190 
million can be achieved. 

Dynamic Contour Environment 

Dynamic contour plots can also be used to depict the technology space, as shown in 
Figure 6. This screen is interactive and has the power of the RSEs behind it. The top 
portion of Figure 6 illustrates the control panel used to manipulate the dynamic contour 
plot. This control panel shows the k-factors that can be adjusted within the specified 
ranges (see Table 4). Any combination of these k-factors can be used to view the 
technology space. This display is set to show subsonic fuel flow versus subsonic drag. 
Therefore, this design space is viewed in terms of the aerodynamics and propulsion 
disciplines. The bottom of the control panel indicates the color-coded responses as well 
as their corresponding limits. The display is shaded with the appropriate color for the 
response that is being violated. Figure 6 shows the baseline settings, meaning that no 
technologies have been infused in this plot. There is no feasible design space without the 
infusion of technologies as evidenced by completely shaded regions which indicate that 
at least one constraint is violated. 



Figure 6: Baseline Contour Plot 
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Feasible space in the contour plots would be indicated by white (or unshaded) space. In 
Figure 6, the dynamic contours have been set to show various gross weight (green) and 
acquisition price (purple) contours. In this way the sensitivity of the system to changing 
limits is seen. By modifying the k-factor settings in the control panel, the design space 
can be explored in real time to determine if the constraints can be met as technologies are 
introduced. The hairlines shown in Figure 6 correspond to the current setting of metric 
constraints. By moving these crosshairs, the current settings are altered, and the potential 
for the system to meet gross weight and acquisition price limits is seen. 

Figure 7 shows a feasible technology space after the current values of the k-factors were 
altered to reflect the application of technologies. The feasible space is represented by the 
unshaded (white) region of the contour plot. 


Hod 2 

Varl 

Factor 

Current X 

r 

r 

SW 

8500 

r 

r 

SVT 

960 

r 

r 

SHT 

1700 


r 

FCDSUB 

0.8582751 

r 


FFFSUB 

0.BB71212 

r 


FRWI 

0.6 

r 

r 

FBFU 

1 

r 

r 

WELEC 

1 

r 

r 

WENQ 

19000 

r 

r 

WHYD 

1 

r 

r 

CfWAL 

1 

: r 

r 

AKOANDS 

-0.15 

; r 

r 

AKFIOTE 

-0.1 

: r 

r 

AKPRICE 

•0.1 

' r 

r 
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Figure 7: Feasible Technology Space 

There is no limit to the number of combinations of k-factor settings that produce this 
feasible design space. Thus, it is incumbent upon the designer to determine which 
technologies to pursue, based on the amount of improvement in each k-factor needed for 
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a feasible and viable design. In this case, the settings chosen in Figure 7 were input to the 
TIF environment. This new TIP can be seen in Figure 8. 



Conclusion 

The design method utilized in this study includes a technology impact forecasting (TIF) 
environment whereby the decision-maker has the ability to easily assess and trade-off the 
impact of various technologies. This technique provides a methodical approach where 
technically feasible and economically viable alternatives can be identified with accuracy 
and speed to reduce design cycle time, and subsequently, life cycle costs. It was achieved 
through the use of various statistical and probabilistic methods, such as Response Surface 
Methodology and Monte Carlo Simulations. This methodology allows for more 
information to be brought into the earlier phases of the design process and will have 
direct implications on the affordability of the system. The increased knowledge allows 
for optimum allocation of company resources and quantitative justification for 
technology program decisions resulting in affordable, high quality products. 
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Nine technologies were infused into the 600-passenger commercial transport concept. 
The Technology Readiness Level (TRL) of each technology was established through a 
literature review of applied research. From the search, the readiness levels were mapped 
to a probabilistic space such that technologies could be infused into the vehicle. 
Physically compatible technology combinations were evaluated and ranked based on the 
improvements to the customer requirements. The technology space investigation showed 
that technologies to decrease the acquisition price and to decrease the gross weight of the 
aircraft were most important. The study also identified three technologies as significant 
for further investigation, specifically composite fuselage structures (T2), aircraft 
morphing techniques (T3), and smart, green engine systems (T6). A concept containing 
these technologies could meet all imposed customer requirements and could create the 
largest feasible design space in which system trade-offs could occur. 
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Task 4: Code Fidelity 
Objectives and Motivation 

In today’s globally competitive aerospace marketplace, economic desirability plays just 
as big a role as technological and performance superiority in capturing the market share. 
Furthermore, the combined effects of budget restrictions and increasing aircraft systems 
costs have caused the aerospace community to shift from a design for performance 
philosophy to a design for affordability philosophy. This has created a growing need and 
interest in the development of effective cost analysis methods and tools. With this as a 
driving motivation, much research has been done at the Aerospace Systems Design 
Laboratory at Georgia Tech in the area of linking sizing and synthesis tools with cost 
estimating tools, such that the overall technical feasibility and economic viability of 
design alternatives can be evaluated during the early stages of design. One such method, 
as outlined in [Ref. 4] involves a Robust Design Simulation (RDS) approach that allows 
for an assessment of risk and uncertainty with regards to performance, cost and schedule. 

The main premise of robust design is the belief that a product should be designed such 
that a desirable range of performance parameters can be achieved even when variations 
are experienced within the operating environment of that product. These variations, 
referred to as noise factors, are considered parameters that are beyond the control of the 
designer but impact the performance of the system. A robust design is then one that is 
insensitive to the uncertainty associated with the noise variables that affect its 
performance. The method developed by ASDL differs from that of traditional design in 
that the objective is to determine a probability distribution for an overall evaluation 
criterion rather than an optimized single point design solution. This is done by allowing 
for variability due to uncontrollable factors (noise variables, economic uncertainty, etc.) 
while evaluating the relative contributions of key product and process characteristic to the 
chosen overall evaluation criteria [Ref. 4]. Using this approach a technologically feasible 
design can be determined and its economic viability evaluated. The difference between 
technical feasibility and economic viability is illustrated in Figure 9. A technically 
feasible design is one that is capable of being produced due to an existing technology 
level. Economic viability is associated with the economic performance of such a concept. 
As shown in Figure 9, a design that is technically feasible is not necessarily economically 
viable. If a design is not economically viable, then a way to shift the mean of the 
response closer to the target must be identified. Therefore, the main thrust of robust 
design is to identify all the critical design variables and technologies, demonstrate the 
effect these variables have on the economic viability of the design, and determine ways in 
which the design can be made more economically desirable. 
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Figure 9: Need To Shift Feasible Design to Economically Viable/Feasible Design 

A key element to this process is the economic analysis tool used to estimate the life cycle 
costs of the product, from research and development through disposal. Due to this 
increased focus on economics, the methods used to perform life cycle cost estimations are 
being critically evaluated. Traditionally, cost estimations take the form of exponential 
equations that have been developed based on historical data from existing systems. 
Ideally, one would want these estimations to be process or activity based. However, this 
type of information is very rarely kept track of in a manner that would make it possible to 
establish adequate relationships. Instead, the estimations are formed based on variables 
of convenience (i.e. system weights, empty weight of vehicle, etc.) or those 
characteristics of the system that statistics are available on. While these relationships are 
adequate for derivatives of existing systems, they become highly unreliable when 
addressing new technologies. In other words, the equations are only valid over the range 
for which they were developed. How well they can predict the behavior of a system that 
performs outside of these initial ranges is unknown. While the need to move to activity 
or process based costing methods has been recognized, due to a lack of sufficient data, 
there have not been significant advances made in this area. Therefore, weight based cost 
estimations continue to be used. However, complexity factors are added to the equations 
in order to allow the user to scale the costs for new technologies according to the relative 
increased complexity. 

The fidelity of these economic codes represents how well the cost estimations capture 
reality. The Cost Estimation Relationships (CER’s) within the economics analysis codes 
usually take the form of an exponential equation: Y = otX®, where Y is the cost and X is a 
regression factor such as gross weight. These equations are developed applying 
regression techniques to existing data. How well these cost estimations represent reality 
depends highly on the scatter of the original data, as well as the fit of the regressed curve 
to that data (see Figure 10). However, the original data is typically considered 
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proprietary and is not available to the public. This requires the development of a means 
to evaluate the fidelity of the codes without access to the original data. 



Figure 10: Example of Data Regression 

It is important to understand the impact that code fidelity can have on the overall design 
alternative evaluation process. The fidelity of the economic code used in robust design 
simulation can potentially negate the benefits of performing an economic uncertainty 
sensitivity analysis. If the variations in system performance due to the infidelity of the 
code are more significant than those caused by economic noise variables then the whole 
idea of design robustness becomes moot. In this circumstance, the variation in system 
performance cannot be unequivocally attributed to the economic noise. Therefore, it 
becomes pointless to determine values for design variables that minimize the impact of 
noise variables. 

A method for evaluating the fidelity of an economic analysis code is presented here. The 
economic code chosen for this case study is Aircraft Life Cycle Cost Analysis (ALCCA). 
The fidelity evaluation is performed probabilistically by using the complexity factors that 
are found within the cost estimating equations in ALCCA to cause a shifting of these 
equations. Each complexity factor is assigned a probability distribution that represents 
the scatter of the original data around the fit curve. Utilizing the NESSUS/Fast 
Probability Integration (FPI) software, the complexity factors are allowed to vary based 
on these distributions, the economic code is run, and the responses of interest are tracked. 
From this information, cumulative distribution functions and sensitivity factors are 
generated using the FPI software. This information can then be used to determine the 
variability in the response that occurs due to shifting of the cost estimating curves, and to 
which specific CER’s each response is most sensitive. 
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Tools implemented 

In order to implement the fidelity study described in the previous section, two simulation 
tools were utilized. The first is the economic analysis code under investigation, which is 
Aircraft Life Cycle Cost Analysis (ALCCA). The second tool is the Numerical 
Evaluation of Stochastic Structures Under Stress/Fast Probability Integration 
(NESSUS/FPI) code, which was used to perform the probabilistic analysis. 

ALCCA 

The roots of the Aircraft Life Cycle Cost Analysis code (ALCCA) can be traced to the 
early 1970s when a series of computer program subroutines were developed to predict 
commercial aircraft return on investment based on engineering economic theory. Later, 
NASA funds would provide the development support for Cost Estimating Relationships 
that would be used with Anderson’s original program to form an extended version of 
commercial aircraft return on investment analysis code. This more sophisticated code, 
developed by Bobick et al in the late 1970s was funded through NASA’s Analysis of the 
Benefits and Costs of Aeronautical Research and Technology program. These models 
were developed in order to analyze the economic viability of applying advanced 
technology to transport aircraft. The original version contained three main modules: 
Fleet Accounting, Airframe Manufacturer, and Air Carrier. These modules, with the 
exception of the Fleet accounting portion, were then used to perform cost estimates in 
ACSYNT a performance and sizing code developed at NASA-Ames. In 1993, the cost 
module was removed from ACSYNT and transformed into stand-alone code that became 
the original version of ALCCA. A number of improvements to ALCCA have since been 
made at ASDL, including a detailed RDT&E (Research, Development, Test and 
Evaluation) cost module that was developed as part of this grant, and will be subject to 
scrutiny in this fidelity study [Ref. 5]. 

The flow of logic and calculations used by ALCCA is shown in Figure 11. First the 
aircraft manufacturing costs are calculated including detailed research, development, 
testing, and evaluation costs. Next the manufacturer’s cash flow and discounted Return 
on Investment (ROI) are calculated for several possible aircraft prices. The aircraft price 
is then based on the manufacturing costs and the rate of return desired by the 
manufacturer. With this price the airline operating costs are calculated including revenue 
loss due to failure and finally, the airline cash flows are calculated and used to determine 
the airline return on investments for several possible values of the yield per Revenue 
Passenger Mile($/RPM). The output file from ALCCA includes values for component 
costs, RDT&E costs, learning curve effects on aircraft costs, manufacturer and airline 
cashflows and return on investments, acquisition price, and direct, indirect, and total 
operating costs. 
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Figure 11: ALCCA Information Flow 


NESSUS/FPI 

The concept of Fast Probability Integration has its roots in structural reliability analysis 
where limit states are used to pre-define failure conditions. This technique possesses a 
multitude of capabilities for computing the probabilistic response or the reliability of 
deterministic models which are governed by uncertain variables. The deterministic 
models can be as simple as an analytical expression for the deflection of a beam or as 
sophisticated as a finite element model. In this study, FPI techniques within NESSUS 
(Numerical Evaluation of Stochastic Structures Under Stress) are used with the cost 
estimation of an aircraft as the deterministic model. 

NESSUS perform a probabilistic analysis on the system responses based on a set of user 
defined random variables, and their corresponding statistics in the form of probability 
distributions. A performance function, and desired probability levels are also required to 
execute an FPI analysis. With this data FPI generates cumulative distribution functions 
and sensitivity factors. 

The two main elements of an FPI technique are the response or performance function and 
the limit state function. The response function, referred to as the Z-function can be 
represented as: 

Z(X) = Z(Xi, X 2 , X 3 , , Xn), where Xi (I = 1, n) represent the random variables. 

The limit state function, also referred to as the g-function is defined as: 
g = Z(X) -Zo = 0, where Zo is a particular value for Z. 

The g-function is defined such that g(X) = 0 defines the boundary between failure and safe 
regions in the random variable space. This is used to calculate the CDF (Cumulative 
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Distribution Function) by varying Zo and computing the point probability. The CDF of Z 
at Zo equals the probability that failure will occur {g<0). 



Figure 12: FPI Inputs and Outputs 


Given the g-function and a joint probability density function PDF, the probability of 
failure can be determined using a standard Monte Carlo procedure of random sampling. 
However, due to the fact that this type of procedure is inefficient for complicated g- 
functions, FPI offers approximate analysis options. Several of these methods are based 
on the concept of Most Probable Point (MPP). The MPP, also known as the design point, 
is defined in u-space which is the coordinate system for an independent, standardized 
normal vector u. The joint PDF (Probability Density Function) is defined in u-space as 
rotationally symmetric around the origin. It decays exponentially with the square of the 
distance from the origin. The transformation of the g(X) function to g(u) allows the MPP 
to become the minimum distance from the origin to the limit-state surface, which means 
that the density is a maximum when the distance is a minimum. The concept of MPP is 
essential to fast probability analysis. For a detailed description of the distribution 
transformation used to transform g(X) to g(u) and the MPP search procedure that is 
implemented within FPI please refer to Reference 6. 

There currently exist 9 methods within NESSUS for performing the probabilistic 
analysis, which are: 

• First-Order Reliability Method 

• Second-Order Reliability Method 

• Advanced First-Order Method 

• Fast Convolution Method 

• Radius-based Importance sampling with radius reduction factor 

• Standard Monte Carlo Method 

• Radius-based Importance Sampling with user-defined radius 
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• Adaptive Importance Sampling Method 

• Mean Based Methods (MV, AMV, and AMV+) 

Of these methods, the Advanced Mean Value (AMV), a mean based method, was chosen 
for this study. The mean based methods are used with complicated g-functions that 
require time intensive calculations. The mean value method (MV) uses an approximate 
g-function that is generated by linearizing g at the mean values of the random variables. 
The advanced mean value method (AMV) takes the MV solution and improves it by 
applying the Most Probable Point Locus (MPPL) of the MV g-function. The AMV+ 
method improves this even further by using the MPPL of the exact g-function [Ref. 6]. 

Method of implementation 

Utilizing the tools discussed in the previous section, the fidelity of ALCCA was 
analyzed. The first step was to identify the cost estimations within ALCCA which would 
be evaluated. The RDTE module (subRDTE.f) and the Manufacturing Cost Module 
(accost.f) were chosen due to the fact that most of the cost estimation equations can be 
found in these two modules. Figure 13 summarizes the breakdown of costs within the 
manufacturing cost module. Within each category there are a number of cost equations, 
as listed next to the category titles in parenthesis. There are a total of 30 manufacturing 
cost estimation relationships (CER’s) that were evaluated. For each of these equations 
the complexity factor was identified and designated as a random variable within FPL 
Each complexity factor was then assigned a distribution. Ideally, if historical data were 
accessible, these distributions would be assigned based on the actual statistics of the data. 
However, since this information is not available for this study, the shape functions for 
each complexity factor are assumed. A normal distribution was assigned to each of the 
manufacturing complexity factors. 


- Wing Group (3) 

- Tail Group (3) 

- Body Group (3) 

- Alighting Gear Group Structure (3) 

- Nacelle Group (3) 

- Propulsion Group (15) 

Figure 13: Manufacturing Cost Approximations Evaluated (30 total) 

For this study, the values of the complexity factors are set to fall within ±30% of the 
mean value (3a = 0.3), varying from 0.7 to 1.3. Hence, each complexity factor has a 
mean of 1.0 and a standard deviation of 0.1 as illustrated in Figure 14. 
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Figure 14; Illustration of Mean and Standard Deviation for Normal Distribution 

These distributions are defined for each of the random variables (complexity factors) 
within the FPI input file. Additionally, the probability levels for which the cumulative 
distribution function should be computed are specified within the FPI input file. Nine 
probability levels are defined for the Cumulative Distribution Functions (CDF): 0.01, 
0.05, 0.1, 0.3, 0.5, 0.7, 0.9, 0.95, and 0.99. As previously stated, the AMV method was 
chosen in order to generate the first unit cost CDF. 

The final element that the user must specify for FPI is the Performance Function or Z. 
For time consuming codes, a relationship can be determined using a design of experiment 
and developing a response surface equation. However, given ALCCA’s fast execution 
time, FPI was directly linked to ALCCA and the value of the desired response was 
tracked. For the Manufacturing Cost approximations, the final aircraft price was used as 
the performance function since variations of the final aircraft price as manufacturing 
complexity factors change is of interest. 

In addition to the manufacturing cost approximations, the RDT&E cost approximations 
(subRDTE.f in ALCCA RDT&E cost module) are also observed in evaluating the fidelity 
of ALCCA. The RDT&E costs fall into 6 main categories as shown in Figure 15. Within 
each of these categories there are a number of subcategories that the cost approximation 
equations are divided into, as listed next to the category titles in parentheses. There are a 
total of 269 cost estimating equations for the RDT&E costs. Due to limitations of FPI 
(which can only handle up to 100 random variables at one time) six separate cases are run 
for each of the six main categories in the RDTE cost. Each of these cases was set up in 
the same manner as was described for the manufacturing cost case. The response tracked 
for these cases was the total RDT&E cost. The same 9 probability levels as were defined 
for the final aircraft price CDF were used for the total RDT&E cost. Similarly, the AMV 
method was used to generate the CDF. 
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Basic Engineering (55) Supplier Non-recurring Material Cost (65) 

Design Engineering Cost ^66^ SPFTi Structures (2) 

SPFTi Structures (2) ^ 
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Laminar Flow System (3) 

Avionics (4) 

Software Engineering 

Operational Flight Software (18) 

Test Programs and Mockups (16) 
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Flight Test Engineering (1) 
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Ground Test Development (5) 

Flight Test Development (1) 

Mockups/EDF Development (2) 

Tooling and Factory Test 
Equipment (35) 

SPFTi Structures (2) 

THPL/other Structures (8) 

Propulsion group (5) 

Fixed Equipment (12) 

Laminar Flow System (3) 

Avionics (4) 

Integration/Assembly (1) 

Figure 15: RDT&E Cost Approximations Evaluated (269 totals) 

In order to streamline the execution of studies such as this one a subroutine named 
RESPON.f was added to the NESSUS code. This subroutine identifies the factors to be 
treated as random variables as well as their probability distribution parameters. Within 
this subroutine a shell script is executed to set up the appropriate input files and run the 
program that is to be used as the performance function, as well as collect the resulting 
response values. Samples of this subroutine and shell script, as well as a more detailed 
description of the FPI input file can be found in Appendix C. 

FPI will generate a number of outputs. In this case the resulting response CDF and 
variable sensitivity factors are of interest. The Z-levels (response values) corresponding 
to the input P-levels (probability levels) can be plotted. The sensitivity factors of each 
random variable listed in the FPI input file can be displayed as bar charts comparing the 
relative influence of each CER considered on the overall response. 

Case Study Results 

Implementation of the study yielded output files (one for each successive run) that 
contain the CDF for each response and the corresponding sensitivity factors. Two main 
observations are made. Firstly, the variability of the two responses (final aircraft price 
and RDT&E cost) due to shifting of the cost estimating curves is observed from the CDF. 
Secondly, the specific cost estimating equations to which each response is most sensitive 
are identified from the sensitivity factors. 
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Manufacturing Cost Module 

Figure 16 shows the CDF for the final aircraft price due to variation in the manufacturing 
complexity factors. With the complexity factors varying ±30% from 0.7 to 1.3, the final 
aircraft price range is captured between $165. 16M (99% probability) and S162.93M (1% 
probability). Overall change of the final aircraft price is calculated to be $2.22M or a 
1.36% total variation from a mean value of $164.09M. Also, the standard deviation is 
calculated to be $0.78M, which means that 68% of the total population falls within a 
mere ±0.48% of the mean. These two characteristics show that the variability is 
relatively small in comparison with the value of the final aircraft price. Hence, it can be 
deduced that fidelity of the Manufacturing Cost module of ALCCA is high. 


Final Aircraft Price due to Variation In Manufacturing Complexity Factors 



Figure 17 below shows the top 10 out of 30 cost estimating equations that the final 
aircraft price is significantly sensitive to. By varying the complexity factors of the 
manufacturing cost module while keeping all others equal, the final aircraft price is found 
to be most sensitive to: 

• Aerodynamic Controls Manufacturing 

• Engine Nacelle Structure Titanium Manufacturing 

• Passenger Accommodations Manufacturing 

• Avionics System Manufacturing 

• Landing Gear Structure Aluminum Manufacturing 

• Landing Gear Structure Titanium Manufacturing 

• Instrument System Manufacturing 

• Empennage Structure Titanium Manufacturing 

• Hydraulic System Manufacturing 

• Electrical System Manufacturing 
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Sensitivity of Manufacturing Cost Complexity Factors 
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Figure 17: Manufacturing Cost Estimations Sensitivity Factors 


RDT&E Cost Module 

As discussed previously, six different cases were run for the RDT&E cost module, each 
representing a different component of the RDT&E cost. Table 6 shows the summary of 
RDT&E cost variability in each component when the corresponding complexity factors 
vary ±30% from 0.7 to 1.3. The largest variability in RDT&E cost is only a little over 
6%, resulting from the variation in Basic Factory Labor components. Meanwhile, the 
smallest variability, 0.74%, occurs when complexity factors in Material Cost component 
are varied. The cost estimating equations that are most influential to the variability of the 
RDT&E cost for each component are listed in Table 7. Individual CDF plots and 
sensitivity factor charts for each RDT&E cost category are shown in Appendix D. 


Table 6: Summary of Total RDT&E Cost Variability for Each Cost Component 


RDT&E Component 

Max. Change ($M) and 
Percentage Change*(%) 

Mean ($M) 

Std. Dev. ($M) 

Basic Engineering 

$291.43M(4.54%) 

$6417M 

$99.2M 

Test Programs & Mockups 

$185.81M (2.93%) 

S6343M 

$63.2M 

Material Cost 

$47.55M (0.74%) 

$6401M 

$16.1M 

Supplier Non-recurring Cost 

$102.97M(1.57%) 

$6555M 

$35.0M 

Basic Factory Labor 

S386.92M (6.08%) 

$6368M 

$131.7M 

Tool & Factory Test Equip. 

$209.58M (3.73%) 

$5620M 

$71.4M 


* Calculated relative to the mean value of RDT&E Cost 
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From Table 6, it can be deduced that all six RDT&E Cost components have relatively 
high degrees of fidelity since the variability of each component is small in comparison 
with the value of the RDT&E cost. However, this deduction is only applicable to each 
component independently since the results tabulated in Table 6 are obtained from six 
independent and separate runs. With that being clarified, no conclusion can be drawn on 
the overall fidelity of the RDT&E Cost module. Hence, a technique that can somehow 
provide an insight on the overall fidelity of the RDT&E Cost module is needed. 

Table 7: Most Influential CER’s for Each RDT&E Cost Component 


Basic Engineering 

- Fuselage Structure Design 

- Surface Controls Design 

- Anti-Icing Design 

- Loading and Handling 
Equipment Design 

- Nacelle Structure Design 

- Pneumatic Equipment Design 

- Electrical Group Design 

- Fault Management and 
Reconfiguration 

- SPFTi Wing Structure Design 

Test Programs & Mockups 

- Flight Testing Engineering 

- Other Ground Testing 
Development 

- Flight Testing Development 

- Ground Testing Engineering 

- Static Ground Testing 
Development 

- Fatigue Ground Testing 
Development 

Material Cost 

- Landing Gear Structure 

- Wing Titanium Structure 

- Fuselage Structure 

- Instruments 

- Seats and Galleys 

- Fuel System 

- Surface Controls 

Supplier Non-recurring Cost 

- Air Conditioning System 

- Weather Radar 

- Surface Controls 

- Instruments 

- Electrical Group 

- Landing Gear 

- EO/TV Surveillance System 

Basic Factory Labor 

- Fuselage 

- Wing Titanium Structure 

- Nacelles 

- Integration/Assembly/Co 

- VG Inlet 

Tooling & Factory Test 
Equipment 

- SPFTi Wing Structure 

- Fuselage Structure 

- Nacelle Structure 

- Integration/Assembly/Co 

- VG Inlet System 

- Electrical Group 

- SPFTi Empennage 


The challenge in obtaining an estimate of overall variability in RDT&E is to overcome 
the main limitation of FPI, which can only handle up to 100 random variables in a single 
run. The approach taken is to screen all 269 RDT&E cost estimating equations that were 
analyzed for the six independent RDT&E Cost module runs in terms of their sensitivity 
factors. Screening and ranking these cost estimating equations based on sensitivity 
factors makes perfect sense from a numerical perspective because these sensitivity factors 
are normalized. Hence, the 99 most influential cost estimating equations out of the 269 
are selected as input random variables for a full FPI run that aims to disclose the overall 
fidelity of the RDT&E Cost module. 

Figure 18 shows the CDF for RDT&E Cost due to variation in the complexity factors of 
these 99 cost estimating equations. With these complexity factors varying ±30% from 
0.7 to 1.3, the RDT&E cost is captured with 99% probability between $6,028. 6M and 
$5,459.7M. The variability of the final aircraft price is calculated to be S568.9M or a 
9.9% total variation from the mean RDT&E cost of $5,744.0M. Also, the standard 
deviation is calculated to be $193. 7M, which means that 68% of the total population falls 
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within ±3.4% from the mean. This 9.9% variability is not particularly small, especially 
when compared to the 1.36% variability for the Manufacturing Cost module shown 
earlier. However, the possibility of an increase in the number of variables as a 
contributing factor to the increasing variability must be considered. Hence, for this 99- 
variables, a 9.9% variability in the RDT&E cost may still reflect a reasonably good 
fidelity for the RDT&E Cost module of ALCCA. 

RDT&E Cost due to Variation in 99 Most influential Complexity Factors 



Figure 18; RDT&E Cost Cumulative Distribution Function 

A comparison is made between the 10 most influential CER’s for the overall RDT&E 
Cost and the 10 most influential cost estimating equations for the six independent 
RDT&E cost components. The purpose of this comparison is to validate the screening 
and ranking of the 269 RDT&E cost estimating equations based on their sensitivity 
factors. Unsurprisingly, 9 out of 10 of the cost estimating equations considered in the 
overall case match the most prominent factors for each of the six sub-cost cases with 
minimal changes in rankings. In fact, as shown below, all top 10 CER’s for the overall 
case are ranked within the top 3 of the each independent RDT&E cost component. This 
comparison further reinforces the validity of the screening technique. The most 
influential CER’s for the overall case are: 

• Basic Factory Labor for Fuselage 

Ranked F' in Basic Factory Labor component 

• Fuselage Structure Basic Design 
Ranked F' in Basic Engineering component 

• SPFTi Wing Structure Tooling and Factory Test Equipment 
Ranked F' in Tooling & Factory Test Equipment component 

• Fuselage Structure Tooling and Factory Test Equipment 
Ranked 2’^ in Tooling & Factory Test Equipment component 
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• Flight Testing Engineering 

Ranked F‘ in Test Programs & Mockup component 

• Other Ground Testing Development 

Ranked 2"‘^ in Test Programs & Mockup component 

• Flight Testing Development 

Ranked 5'^'^ in Test Programs & Mockup component 

• Basic Factory Labor for Wing Ti Structure 
Ranked 2"^ in Basic Factory Labor component 

• Supplier Non-recurring for Air Conditioning System 
Ranked F‘ in Supplier Non-recurring Cost component 

• Supplier Non-recurring for Weather Radar 

Ranked 2"'^ in Supplier Non-recurring Cost component 


As shown in Figure 19 wing and fuselage related CER’s dominate the RDT&E costs 
estimates. Note that engine development costs would dominate the RDT&E variation if 
they were included. However, in this case the engine is considered to be a purchased 
item with a fixed input price which includes a share of development costs as well as the 
engine manufacturer’s profit margin. 


Sensitivity of RDT&E Cost Complexity Factors 



Figure 19; RDT&E Cost Estimations Sensitivity Factors 
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Fidelity vs. Noise 

The motive for this study is to observe whether the impact of code infidelity overshadows 
the impact of economic noise variables on the variability in system performance. Thus, 
the study would not be complete without comparing the variability due to the changes in 
CER’s, to that caused by changes in economic factors such as learning curves and labor 
rates. The results from this technique would determine if using ALCCA for Robust 
Design Simulation (RDS) is valid. The following are the key assumptions made: 

i. The baseline complexity factors (1.0) are assumed to capture reality 
conditions with a high degree of fidelity. 

ii. The identified economic uncertainties are assumed to have significant impact 
on the variability of the responses. 

iii. A similar 3-Sigma, normal distribution design is desired for characterizing the 
distribution of the economic uncertainties. 

The next step is to identify the possible economic uncertainties that may significantly 
impact the variability of the responses (Final Aircraft Price and RDT&E Costs). After a 
series of screening and test runs, the variables listed in Table 8 and Table 9 are identified 
as the economic uncertainties that are most influential to the variability of the final 
aircraft price (for Manufacturing Cost module) and the total RDT&E cost (for RDT&E 
Cost module) respectively. As mentioned in assumption iii above, all variables are 
normally distributed over a 0.1 standard deviation over their respective baseline values 
except for variable $CMAN NFV, that is, the number of flight test vehicles. Since this 
variable is discrete, it cannot be an input random variable in FPI. However, NFV is one 
of the most influential variables amongst the other economic uncertainties. Hence, NFV 
is manually changed to have the values of 1, 2 (baseline value), and 3 for each FPI run. 


Table 8: Economic Uncertainties for the Manufacturing Cost Module (14 Variables) 


Economic Uncertainties 

Description 

Baseline Values 

$CMAN FEE 

Manufacturer Fee (%/100) 

0.05 

$CMAN RTRTN 

Manufacturer Return of Investment (%) 

12.0 

$CMAN RTRTNA 

Airline Return of Investment (%) 

10.0 

$CMAN RE 

Engineering Labor Rate ($/hr) 

85.0 

$CMAN RT 

Tooling Labor Rate ($/hr) 

55.0 

$CMAN LEARNl 

Airframe Learning Curve Factor for 1 Lot {%) 

82.0 

$CMAN LEARN2 

Airframe Learning Curve Factor for 2"^^ Lot {%) 

82.0 

$CMAN LEARNAl 

Avionics Learning Curve Factor for 1*' Lot (%) 

82.0 

$CMAN LEARNA2 

Avionics Learning Curve Factor for 2"^ Lot (%) 

82.0 

$CMAN LEARNASl 

Assembly Learning Curve Factor for Lot (%) 

82.0 

$CMAN LEARNAS2 

Assembly Learning Curve Factor for 2"^* Lot (%) 

82.0 

$CMAN LEARNFEl 

Fixed Equipment Learning Curve Factor for 1*^ Lot (%) 

82.0 

$CMAN LEARNFE2 

Fixed Equipment Learning Curve Factor for 2"*^ Lot (%) 

82.0 

$CMANNFV 

Number of Flight Test Vehicles 

2.0 
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Table 9: Economic Uncertainties for the RDT&E Cost Module (23 Variables) 


Economic Uncertainties 

Description 

Baseline Values 

$RDTE FLTHRS 

Flight Hours for Flight Testing (hr) 

6000.0 

$RDTE CMATFLTR 

Cost of Flight Test Material per Flight Hour ($/hr) 

8500.0 

$RDTE RMFGMAT 

Rate for Manufacturing Material Cost 

1.04 

$RDTE RTENGMHR 

Cost per Test Engineering Manhour (S/MHR) 

86,51 

$RDTE RDEVPMHR 

Cost per Development Manhour ($/MHR) 

50.64 

$RDTE RMANUMHR 

Cost per Manufacturing Manhour (S/MHR) 

50.64 

$RDTE RMANSUP 

Cost per Manufacturing Support Manhour (S/MHR) 

54.86 

$RDTE RQA 

Cost per Quality Assurance Manhour (S/MHR) 

56.97 

SRDTE RLOG 

logistic Rate, S/ILS Manhour (S/MHR) 

82.29 

$RDTE RPMGT 

Cost per Management Manhour ($/MHR) 

94.95 

$RDTE LEARNTIM 

Airframe Material LC for Ti Structure for 1“ Lot/Prototype (%) 

95.0 

$RDTE LEARNM 

Airframe Material LC for other Mat’l for 1*' Lot/ft'otolype (%) 

95.0 

SRDTE LEARNPRM 

Propulsion System Material LC for P' Lot/Prototype (%) 

93.5 

SRDTE LEARNENM 

Engine Material LC for 1*‘ Lot/Prototype (%) 

93.5 

$RDTE LEARNFEM 

Fixed Equipment Material LC for P' Lot/Prototype (%) 

93.5 

SRDTE LEARNAAM 

Avionics Group A LC for P' Lot/Prototype (%) 

93.5 

SRDTE LEARNABM 

Avionics Group B LC for 1“ Lot/Prototype (%) 

92.0 

SRDTE LEARNLFM 

Laminar Flow Control Material LC for I Lot/Prototype (%) 

93.5 

SCMAN FEE 

Manufacturer Fee (%/I00) 

0.05 

SCMAN RTRTN 

Manufacturer Return of Investment (%) 

12.0 

SCMAN RE 

Engineering Labor Rate (S/hr) 

85.0 

SCMAN RT 

Tooling Labor Rate (S/hr) 

55.0 

SCMAN NFV 

Number of Right Test Vehicles 

2.0 


The noise related uncertainty study is carried out in a manner similar to the fidelity 
analysis. The noise factors identified as significant for manufacturing and RDT&E costs 
are designated as random variables within FPI, the automated link between FPI and 
ALCCA is executed, and the resulting response is tracked. Figure 20 and Figure 21 
display the response CDF’s that result from the uncertainty defined for the economics 
factors. In this case the sensitivity factors, although generated, are not as interesting, 
since the economic factors to be varied were already chosen based on their expected 
influence. 
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Final Aircraft Price due to Variation In Economic Uncertainties 



RDT&E Cost due to Variation In Economic Uncertainties 



RDT&ECost (SMIL) 


Figure 21: CDF for RDT&E Costs Variation due to Economic Uncertainties 

Table 10 and Table 11 compare the variability and standard deviation caused by code 
infidelity with that caused by the noise factors (economic uncertainties). The data in 
these tables clearly shows that the economic uncertainties caused significantly more 
variability to the system performance than code fidelity. This implies that RDS can be 
successfully implemented using ALCCA without worrying about the impact of code 
fidelity overshadowing the changes due to economic uncertainties 


34 




Task 4: Code Fidelity 


Table 10: Variability Caused by Code Infidelity vs. by Economic Uncertainties 



Percentage Change relative to Mean Cost 


Fidelity Investigation 

Noise Var. 

% Difference 


Run A 

Run B 

[100*(B-A)/A] 

Manuf. Cost Module 

1.36% 

3.80% 

180.02% 

RDT&E Cost Module 

9.90% 

16.01% 

61.62% 


Table 11: Comparing Standard Deviation of Variability Caused by Code 
Infidelity and by Economic Uncertainties 



Standard Deviation from Mean Cost 


Fidelity Investigation 

Noise Var. 

% Difference 


Run A 

RunB 

[100*(B-A)/A] 

Manuf. Cost Module 

$0.78M 

$2.06M 

166.1% 

RDT&E Cost Module 

$193.7M 

$333.5M 

72.23% 


Graphically, the difference in the way that these two sources of uncertainty influence 
overall costs can be seen in Figure 22. This figure shows a shallower, but wider CDF 
resulting from the economic uncertainty study as is expected given the differences in 
standard deviations. Furthermore, the mean cost is also shifted higher due to the 
exponential nature of cost relationships. The effect of the factor raised to a power in an 
exponential equation is magnified when it increases, and diminished when it decreases, 
whereas the complexity factors used in the fidelity study linearly influence cost. 
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Final Aircraft Price Variability Due to Noise and Code Rdeiity 



Figure 22: Comparison of Noise vs. Fidelity Uncertainty 


Conclusions 

The results obtained in this exercise lead to several conclusions. Firstly, the final aircraft 
price demonstrates a very small change with respect to the mean (1.36% or $0.78M 
standard deviation). Therefore, Manufacturing Cost module of ALCCA can be 
considered to have a high degree of fidelity. 

Secondly, the variability and standard deviation of each independent component of the 
RDT«&E Cost Module of ALCCA are relatively small in comparison with the value of the 
RDT«&E cost. The outcome of the FPI run that encapsulates the impact of 99 most 
influential cost estimating equations on the variability of overall RDT&E cost showed 
promising results. Comparison between the 10 cost estimating equations that RDT&E 
cost is most sensitive to for both the 6 independent component runs and the 99-variables 
run validates the technique of screening and ranking the 269 RDT&E cost estimating 
equations based on their sensitivity factors. RDT&E cost was shown to have a 9.9% total 
change with respect to the mean while the standard deviation was $193.7M. Taking into 
strong consideration the contributory impact of the increased number of variables for this 
99-variables FPI analysis, it is concluded that the 9.9% variability may still reflect a 
reasonable overall fidelity for the RDT&E Cost module of ALCCA. 

Thirdly, under similar assumptions and probabilistic settings as the fidelity investigations, 
the comparison of fidelity results to economic variability showed that the variability 
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caused by economic uncertainties clearly surpasses the variability caused by code 
infidelity. This implies that using ALCCA to perform RDS in order to reduce the 
sensitivity of a design to uncontrollable economic factors is indeed a valid option. 

Based on the relatively small variabilities observed in the RDT&E and Manufacturing 
Cost modules, and backed by the comparison between code fidelity and economic 
uncertainties, the overall conclusion drawn for this study is that these two main modules 
of ALCCA have shown reasonably high degrees of fidelity. This implies that the use of 
weight-based cost estimation relationships (CER’s) in ALCCA still provides accurate 
cost estimates for existing aircraft with infusion of derivative technologies. Nevertheless, 
the motivation for shifting to process and activity based costing still exists in the 
aerospace community in order to be more accurate and versatile when estimating costs of 
future aircraft where revolutionary technologies and concepts will refine the science of 
aviation. 
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Capacity Focus Task: Formulation of a Method to Assess 
Technologies for the Improvement of Airport Capacity 

The globalization of the worldwide economy, coupled with airline deregulation and trade 
expansion, has caused a boom in air travel. This rapid growth has not been paralleled by 
a similar expansion in the national airspace infrastructure, resulting in congestion, delays 
and widespread frustration. The problem is quickly reaching gridlock proportions and 
the pressure for solutions is increasing. However, the National Airspace is not a flexible 
system. Solutions implemented today will only be felt in the long term. These solutions 
will also require a significant capital investment for system-wide execution. Thus, a 
careful process to determine which solutions will provide the highest payoff with the 
lowest risk is essential. The development of such a process and the modeling 
environment that it relies on are the goals of this research. 

Motivation 

Increasing Demand 

Norman Mineta, the Secretary for the Department of Transportation, tells us: “The only 
sure remedy for air traffic control congestion in the near term would be a recession, 
which would suppress demand” [Ref. 7]. Indeed, it is the growth in demand beyond the 
system capabilities that has caused today's congestion. However, the increase in air 
travel has been a direct consequence of economic well-being, and has also resulted in a 
better quality of life. Thus, it is important to find ways to accommodate the demand 
generated and alleviate congestion. 

As Mr. Mineta mentioned there is a direct relationship between economic health and 
demand for transportation. In fact, the relationship between GDP (Gross Domestic 
Product) and air travel is one widely recognized and often used to estimate future 
demand. However, the GDP does not explain air travel demand completely, as the 
percentage growth in RPK (Revenue Passenger Kilometers) slightly outpaces the growth 
in GDP, see Figure 23. Lower airline rates following deregulation, nearly 40% cheaper 
than those prior to the Deregulation Act, may account for a portion of that disparity. The 
convenience of today's air market, with multiple departure times and numerous non-stop 
flights, may also have contributed to an increase in demand beyond that justifiable by 
economic growth. However, as the airlines strive to provide customer satisfaction by 
scheduling more frequent flights at convenient times, they may also be aggravating the 
congestion problem. 
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Growtii Varies by Regien 


Amual growth dv cbmlc^e, 1906-1959 



Figure 23: Annual GDP and RPK growth by Region [Ref. 8] 

Just as demand is one of the main ingredients of today's plaguing congestion, the 
distribution of that demand has a very important role to play in the capacity deficit that 
the National Airspace System (NAS) is experiencing. Looking at Figure 24, obtained 
from the OAG (Official Airline Guide) schedules, one can see that operations at a hub 
airport like Atlanta Hartsfield International Airport are not even remotely uniform. It is 
also interesting to observe that a bank of arrivals is immediately followed by a bank of 
departures, possibly reflecting today's interconnectivity of flights. It is obvious from these 
figures that airline scheduling to maximize passenger convenience and utilization of its 
aircraft has resulted in peaks of arrivals and departures at certain times of the day. The 
number of operations at these peak times often approaches or surpasses the capacity 
limits of major U.S. airports. Thus, it is at these times that the worst delays are generally 
recorded. 



Figure 24: Annual Operations at ATL per Hour 
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Those delays are likely to continue getting worse, even considering current plans for 
capacity increases, due to the forecasted growth in both the airline and the freighter 
market. Based on GDP growth and the other factors mentioned, major aircraft 
manufacturers are predicting an average annual growth in world air travel demand of 5% 
over the next 20 years. The increase in e-commerce and world trade will result in an 
even higher growth rate, approaching 6%, for the world's air cargo market. These growth 
rates, when translated to the number of aircraft required to service the demand, mean that 
the world's fleet will double by the year 2019, both in terms of airliners, and in terms of 
freighters [Ref. 8 and 9]. If the trend to increase flight frequencies for customer 
convenience continues, a large portion of that forecasted demand may be serviced with 
single-aisle and regional aircraft, as shown in Figure 25 . Such a trend, very prevalent in 
the United States, would pose an even larger threat of collapsing the NAS [Ref. 10]. 

SinglvAisleAiiplanes Doninate 

Ritoe Dallvorlw 

m brraiiBi re9ona ^eis 

Q Twn-aijk 

H 747anciafCP9r 


Number of airplanes 
'In T!99doeiirt 22,315 

Figure 25; Forecasted 2019 Fleet Breakdown by Size [Ref. 8] 

In 1997 a report from the National Civil Aviation Review Commission warned of the 
impending gridlock [Ref. 11]. All signs indicate that that gridlock is here today, and it is 
here to stay. As the Secretary of Transportation put it in his recent speech to the senate: “I 
need to be very candid with you in this point — we are very likely to have similar, or 
worse, delay problems this year as well” [Ref. 7]. Therefore, changes to the NAS to 
accommodate current and future demand must be made now, for the sake of their future 
benefits. The capacity of the system must grow and keep pace with demand lest it 
hamper transportation and the economic well-being that it signifies. 

Limited Capacity 

The steady increase in air travel demand in recent years has pushed the National Airspace 
System to its limit, but it is the inability of the system to expand accordingly that has 
caused today's delays. Indeed, with the system operating so close to its maximum 
capacity it is not unusual for a relatively small event, such as a local thunderstorm, to 
cause widespread delays, far beyond the area affected by the weather. An example of 
such an occurrence, where an unexpected hold of aircraft in the Newark airspace for 5 
minutes affected 250 aircraft throughout the East and Midwest in less than 20 minutes, 
was compared by reporters to the behavior of a virus spreading uncontrollably [Ref. 12]. 
It's like walking a high wire, the slightest disturbance can have disastrous results. 
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Since flights are generally not scheduled to surpass the capacity of the airports they serve, 
one must assume that full theoretical capacity is not being achieved. It has been 
suggested that the first step toward a reduction of delays is a better management of the 
existing capacity. Having a uniform distribution of demand, or transporting more 
passengers per departure would be relatively easy to implement short-term solutions. 
However, as described in the previous section, the airline profitability could suffer from 
such regulations. Furthermore, better capacity management would not solve the long- 
term congestion of the system. Including all currently planned runway construction as 
potential capacity, the system is currently operating at 57% of its maximum capacity, and 
by 2010 it will be operating at 70% capacity if no expansion policies are implemented. 
Considering that significant delays start occurring at 40% capacity utilization, and grow 
exponentially from there, the delays experienced thus far may be only the tip of the 
iceberg. [Ref. 13]. 

Airport capacities are often described in terms of Pareto frontiers which show a boundary 
curve of arrivals vs. departures similar to the one shown in Figure 26. These curves 
depend on a variety of factors such as runway configuration, safety separations, 
equipment at the Air Traffic Control (ATC) center, navigation aids within the aircraft and 
the weather conditions. However, these curves are not sufficient to characterize the 
capacity of the system. The sectors that are crossed between arrival and destination 
airports also have a limited capacity dictated by human and technical factors. In the 
United States capacity at the airports is generally more limited and therefore considered 
the constraining factor. However, there are a few exceptions like the airspace around 
New York City and Chicago, and in Europe the en-route capacity poses a significant 
concern. 



Figure 26: A Pareto frontier Example 


It could be argued that to properly model an airport these Pareto Frontiers are also 
insufficient. Airside congestion is not the only problem plaguing major airports in the 
United States. Baggage handling, immigration and customs facilities, airport access 
roads and parking lots also have a limited capacity. As the airspace becomes crowded so 


41 




Capacity Focus Task: Formulation of a Method to Assess 
Technologies for the Improvement of Ai rport Capacity 

does the terminal demonstrating why system growth is not easy to implement. The 
complexity and interdependency of all the airspace system components makes changes 
difficult, costly and lagging. 

Complexity of the System 

The need to increase capacity is pressing, but ATC budgets are limited, airport 
communities resist expansion, noise regulations restrict approach and departure paths. 
The solution to the capacity problem is not a simple one due to the complexity of the 
NAS and its different, and often conflicting, interests. 

Airline strategies often conflict with airport and ATC concerns. As an example, the 
recent increase in regional jet departures at peak times in the La Guardia airport, and the 
resulting delays, have forced the airport authorities to impose a limit on the number of 
flights that can use the airport at those times of the day [Ref. 14]. A very large aircraft, 
such as the one currently being developed by Airbus Industrie could serve as a further 
example of the interdependency within the NAS. Such an aircraft would increase the 
demand served per departure alleviating the capacity constraint. However, it would have 
to meet airport restrictions in terms of runway and gate dimensions, it would have to 
follow ATC instructions in terms of required separation with other aircraft, and it would 
have to overcome the current airline tendency to favor smaller more frequent flights [Ref. 
15]. Unfortunately, when analyzing solutions to the NAS congestion problem, 
researchers often focus on a single aspect of the problem, without thoroughly considering 
the effects a change in one of the NAS components will have on the other pieces of the 
air transportation puzzle. As the Secretary Of Transportation puts it: “There must be 
more synchronization and more coordination between these groups if we are going to 
solve the [capacity] problem” [Ref. 7]. Figure 27 illustrates the four main components of 
the NAS, and adds three considerations which are vital when assessing capacity and 
throughput technologies: Safety, Environment and Economics. 
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Figure 27; Components of the NAS 


Economics drive the demand which has made the system capacity inadequate. As 
described earlier, the GDP is a clear indicator for air travel demand. Economics also 
drive the search for solutions to the delay problem as airlines lose revenue and passengers 
lose time. Last year 450,000 flights were delayed 15 minutes or more beyond the delays 
already built in to airline schedules. These delays are very costly to the airlines, both in 
terms of customer satisfaction costs and also in terms of the ripple effects throughout 
their schedule. When a flight into an airline hub is delayed the many passengers who 
have missed their connections must be accommodated with hotels, meals or 
transportation with other airlines. Added to that are the extra labor and airport facility 
costs. But it doesn't stop there; it is likely that the delayed flight was scheduled to 
continue on to some other location, and so the delay spreads. Delay costs for an early 
morning flight are often multiplied by four to account for the effect that will be felt 
throughout the day. Furthermore, economics dictate budgets which limit the range of 
solutions that can be implemented. When considering potential solutions to the delay 
problem, one must also consider whose budget the funds for the project will come from, 
and who will reap the benefits of the improvement. Airport improvements often require 
substantial capital investments that the airport authorities translate into increased landing 
fees for the airlines, who, if the market will withstand it, proceed to pass those costs on to 
the travelers. However, the bulk of capacity improvement technologies are being 
researched by government organizations such as NASA and the FAA (Federal Aviation 
Administration) for the sake of economic well-being and voter satisfaction. Indeed, 
public pressures are a very important factor to contend with. 

The environment in terms of governmental and community pressures can influence 
airport capacity greatly. Community noise has become an increasing concern in the 
neighborhoods surrounding major airports to the point that arrival paths are being 
diverted to avoid populated areas, with the subsequent efficiency loss [Ref. 16]. On the 
other hand, the community needs for air travel can prompt government action, such as the 
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recent approval of the AIR21 bill, that provides funds for airport improvement and the 
inclusion of runway independent aircraft in ATC procedures [Ref. 17, 18, 19]. Indeed, it 
has been the widespread dissatisfaction of the traveling public that has encouraged recent 
Congress probes of the delay problem and has stayed potential aviation budget cuts [Ref. 
12]. As it stands today the FAA budget intends to cover the improvement of ATC 
equipment so as to alleviate the congestion of our airspace. This effort is to include 
research into alternate approaches such as Free Flight. The FAA will also provide grants 
for airport expansion and improvement, not only for capacity enhancements, but also to 
mitigate noise and increase safety [Ref. 20] 

Safety can be viewed as a capacity constraint. It is safety that dictates aircraft separation 
on arrival, a major traffic volume limitation. It is also safety that prescribes bad weather 
procedures further straining system capacity. New technologies designed to relieve 
congestion will not be implemented unless they demonstrate a good safety record. Even 
more, safety must be improved if capacity is to increase; today's accident rates would 
result in a major accident occurring every three days at 2005 demand levels [Ref. 21]. 
Indeed, the goal of the FAA is to reduce today's fatal accident rates by 80% with a good 
portion of their budget dedicated to that purpose [Ref. 20]. Thus, an assessment of 
capacity and throughput technologies without due consideration to safety issues would be 
incomplete. 

A thorough analysis of the NAS is further complicated by the variability it is subject to. 
The economic environment can fluctuate widely with periods of economic boom 
alternating with phases of recession, influencing GDP values accordingly, and affecting 
not only the demand for air travel, but also the revenue yield that can be obtained without 
loss of market-share. Furthermore, many of the day-to-day costs in the NAS are driven 
by factors beyond an analyst's control, such as OPEC fuel production levels or labor 
union agreements. Government policies, often driven by electoral polls, can also have a 
great influence on the funds available for ATC improvements. These factors influence 
both demand and capacity, but capacity is even more deeply affected by weather, which 
judging by the forecasts is rather unpredictable. The inherent uncertainty in the system 
alone would justify a statistical approach to the capacity problem, yielding results in 
terms of probabilities, rather than deterministic values. But an additional degree of 
imprecision is also introduced by the fidelity of the modeling codes used, as accuracy is 
traded off with model efficiency and technologies push the system beyond existing 
databases. Furthermore, the forecasted impacts of technologies which are still in the 
development stage are often not entirely reliable, and a probability associated to the 
potential improvement is often preferred to a deterministic impact prediction. This 
uncertainty in the potential effect of a technology also applies to its negative impacts, 
which are often overlooked or not researched as thoroughly. The complexity and 
uncertainty inherent to the NAS, coupled with the varied nature of the capacity improving 
technologies that have been proposed or are under investigation, clearly establishes a 
need for a statistically based method to assess those technologies from an overall-system 
point of view. 
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Subtask 1 : Capacity Model Analysis 

The first step toward assessing capacity and throughput technologies is to attain an 
understanding of the factors that influence capacity in the terminal airspace. To 
accomplish this, a capacity code developed by the Logistics Management Institute was 
used within a Response Surface Methodology such as that described in Reference 4. This 
methodology uses statistically based sampling to reduce the number of cases required to 
ascertain the influence of factors, and their interactions, on a response of interest. In this 
case the response of interest is capacity under different weather conditions and the factors 
varied include approach separation, runway occupancy time and fleet mix among others. 

Initially the Atlanta capacity model created as part of the Aviation System Analysis 
Capability was executed according to the variable ranges described in Table 12 and a 
suitable 2-level Design of Experiments (DoE) aimed at identifying those variables with 
the most influence on capacity. Unfortunately, upon investigation of the results obtained 
it became apparent that the Atlanta capacity code accessible online [Ref. 22] contained a 
small error since the Runway Occupancy Time (ROT) defined for departures was 
appearing as a significant factor in arrival capacity, in spite of the fact that the Atlanta 
airport generally operates two dedicated departure and two dedicated arrival runways. 
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Table 12: Variable Ranges for Airport Capacity Model 


Upon consultation with LMI, a revised capacity code, albeit for a single runway 
simplified case, was obtained and a new design of experiments using the previously 
defined variable ranges was executed. In this case, each of the capacity points generally 
used to define a Pareto frontier were tracked: Max. Arrivals (A), Max. Departures (D), 
Balanced Arrival and Departures (E), and Free Departures between arrivals (F). The 
variables listed in Table 13 were found to be the most influential for each category. As 
expected, departure related variables such as a departure path and speed dominated the D 
response, whereas approach separation and ROT appeared as most influential for the A 
response. Free-departures and Equal-use capacities showed a mix of the variables 
identified for A and D, and the fleet mix related variables had a pervading influence on 
all responses tracked. 
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Table 13: Most Influential Capacity Variables 
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The variables identified as most influential were then used to create a Response Surface 
Equation (RSE) that approximated the behavior of the code itself. A quadratic response 
surface equation can be created from a statistical analysis of data gathered in the 
execution of a three-level DoE with the ranges previously defined and a center point 
calculated for each variable. This RSE can then be used within a dynamic environment 
called a prediction profile to quickly assess the effect that changing one of the variables 
of interest would have on any of the responses tracked. A sample of one such prediction 
profile can be found in Figure 28. 
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Figure 28: Prediction Profile for Max. Departures in VMCl Weather conditions 

The fit for the departure curves was excellent, since departure capacity is governed only 
by the time required to take off and clear the runway. For approaches and mixed-use 
runways the fit of the model was good, but validation tests indicate that the RSE 
generated is only valid when one constraint dominates. Approach capacity is governed 
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by two main constraints Miles-in-Trail (MIT) and ROT. RSE’s are continuous, therefore 
they cannot capture the discontinuity in capacity behavior caused by a sudden change in 
dominant constraints, therefore in the region where one constraint gives way to the other, 
the RSE is not as accurate an approximation as would be desired. A possible way to get 
around this limitation of RSE’s is to track the capacity dictated by MIT and ROT 
separately, rather than letting the code select the lowest of the two for its output. 
However, this would require additional modification of the LMI code, and the purpose of 
this task, which was to identify the main factors affecting airport capacity, has already 
been accomplished. If this RSE were necessary for integration into a larger environment 
further exploration of the constraints would be necessary, but the execution speed of the 
LMI capacity code is such that the full code can be used in the NAS simulation 
environment without need for an approximate RSE. 

Subtask 2: Modeling the National Airspace System 

The need for a comprehensive NAS model that places aircraft within airline fleets, and 
those airlines within a competitive environment, under airport and ATC restrictions, in 
order to assess capacity and throughput technologies has been established. It has been 
further determined that such a model must also include economic, safety and 
environmental impact assessments. And the model must be versatile enough to accept a 
statistical treatment of the variability existing within the NAS. However, in order to 
create such a model, a through understanding of the NAS and its components is 
necessary. 

The National Airspace System 

The United States contains over 18,000 airports, 3,304 of which are considered part of 
the national system. More than 450 of these are considered primary airports rating an 
FAA control tower to direct traffic during landing and takeoff. Airports provide a 
gateway for air carriers to serve their customers, while ATC provides a framework for the 
safe flight of their fleets. Thus, when considering problems related to the NAS, one must 
consider what air travel demand needs to be served, what aircraft will be chosen to serve 
that demand, how the airline will operate the aircraft, and what infrastructure will be 
necessary for a safe and efficient flight environment. 

Passenger Demand 

Looking up at the night skies over any major city and observing the many lights of 
aircraft flying overhead, one cannot doubt that a demand for air transportation exists. But 
the driving forces behind that demand, and the factors that make a passenger or a cargo 
forwarder choose a particular carrier at a particular time from a particular airport have 
been the subject of extensive research. 

Air travel demand and economic prosperity are closely tied. GDP forecasts are often 
used to determine future demand for air travel. At times GDP data is combined with 
other economic indicators such as unemployment or income per capita to determine the 
availability of expendable income in a particular location. This is done based on the 
assumption that travel, and specially air travel, is considered a luxury. As such, air travel 
traditionally included many services not directly related to the transportation aspect, and 
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catered to a limited demographic group. However, with the advent of deregulation and 
fares that were no longer determined by costs, but rather by competition, fares dropped 
and airlines were forced to reconsider who their customers were. 

Today's airline demand studies often require detailed demographic data considering the 
age and gender of their customer along with factors such as the purpose of the journey 
[Ref. 23]. This enables airlines to tailor their product to meet a wide range of customer 
demands, from the business traveler that seeks comfortable and timely service, to the 
student traveling for spring break that is willing to fly stand-by if that will lower the 
ticket cost. 

Passenger demand is indeed very varied in nature. The minimum requirements could be 
efficient, safe service at a reasonable price, but what is considered a reasonable price or 
efficient service is not set in stone. Thus, the business traveler who puts emphasis on 
comfort and convenience would be willing to pay a higher price, but demands more 
services, for the same seat that an individual on vacation would rather pay less and do 
without the niceties of meals and in-flight entertainment. And an airline must be able to 
serve both to maintain profitability. While the business travelers make up only 10% of 
the total number of passengers, they account for about 40% of all the trips, as well as a 
large portion of the overall passenger revenues. However, an airline cannot cater only to 
this type of passenger since it must meet a minimum load factor (% of seats filled) to 
break even. Thus, 90% of all tickets sold in the United Stated are deeply discounted in 
order to undercut the competition, and fill one more seat, which might make the 
difference between earning a profit or recording a loss for that flight [Ref. 24]. 

Airline Characterization 

Airlines provide a service for their customers transporting them and their belongings 
from one point to another for an agreed price [Ref. 24]. This service orientation is what 
makes the airline business so susceptible to variation in customer demand. Diverse 
customers and the perishability of the airline product, a seat in a particular itinerary at a 
particular time, dictate the ever-changing prices and fierce competition found in today's 
deregulated market. 

The competitiveness of the airline market is further complicated by minimal product 
differentiation and soft brand loyalty. Airlines tend to fly similar routes at similar times 
with similar equipment and service and often matching prices. Frequent flyer programs 
have been relatively successful in introducing a type of brand loyalty, but one that is 
easily put aside for a better price or a more convenient schedule. This is why revenue 
management and scheduling have become such important factors in airline operations 
[Ref. 25]. 

Revenue management allows the airline to price seats, even within the same class, 
differently depending on time of purchase, competition, season or day of the week. This 
allows the airline to capture a larger share of the market as shown in Figure 29. Yield 
management has also given rise to the frequent practice of overbooking to ensure as 
many seats as possible in any given flight are filled in spite of last minute cancellations or 
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no-shows. These practices are often distasteful to the customer, but they have allowed 
airlines to increase their average load factors, thereby increasing the potential for 
profitability and reducing losses due to unused inventory. 
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Figure 29: The Effect of Yield Management 


Airline scheduling practices can also greatly influence their appeal to the customer and 
therefore their revenue stream. Business customers will be more attracted to an airline 
with multiple flights in their itinerary, closely matching their desired departure time, and 
offering them the flexibility of a later or earlier flight if their plans change. Furthermore, 
frequent flights will provide connection opportunities generating additional demand that 
would otherwise not have existed for that city pair. However, the airline desire to 
increase flight frequencies is countered by the cost of increasing carrier capacity and the 
complexity of combining crew schedules, aircraft availability and maintenance needs. 


New aircraft cost millions of dollars making the airline industry very capital intensive. 
These high acquisition costs are coupled with long order times that have wreaked havoc 
on airline economics. Airlines have often ordered new aircraft at times of economic 
growth when the demand would support increased capacity, but have received them when 
the economy was slowing down and demand required downsizing not expansion. This 
has prompted the airlines to place a lot of emphasis on fleet selection weighing the costs 
of a new aircraft, as compared to the maintenance requirements and higher fuel 
consumption of older, but cheaper or already owned aircraft. In some cases airlines have 
also chosen newer aircraft, or newer engines, for their compliance with new noise and 
emissions regulations. Such a decision capitalizes on the additional fuel and maintenance 
savings of newer equipment, rather than using the less expensive, but shorter term, 
solution of a retrofit. Oftentimes, a new mission with a longer range or faster speeds can 
be behind the purchase of new equipment. Aircraft with longer range capabilities open 
the possibility of serving longer non-stop routes. Faster aircraft, such as supersonic 
transports or business jets, can increase utilization, thus affecting the number of trips per 
day an aircraft can cover, as well as addressing the issue of passenger value of time. 
Aircraft size and seating configurations are also factors to consider in fleet planning. 
Larger aircraft can accommodate more passengers per operation, thus increasing capacity 
without a negative effect on airspace congestion or an increase in delays. However, the 
extra seats must be filled to cover the higher cost per operation. Denser seating 
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configurations can also increase the revenue potential per operation without increasing 
operating costs, however, passengers tend to object to cramped flying conditions. All 
options to serve additional demand have one common danger, the potential oversupply of 
seats. While additional seats present a revenue opportunity, the need to fill those seats to 
cover costs may lead the airline to lower ticket prices resulting in little overall profit. 

Airlines, like any business, seek to maximize profit, but they are often caught between 
the finicky consumer that dictates market share and revenue, and costs beyond their direct 
control. This results in net profit margins in the order of 1 to 2% compared to a 5% 
average for the US industry [Ref 24]. Southwest Airlines has found a recipe to solve this 
problem by offering a simplified product, point-to-point service with fast turn-around 
times, and dramatically reduced overhead costs. Unfortunately, traditional airlines, fruit 
of a previously regulated market, have a history of very strong unions that capitalize on 
the intensive training required for pilots and maintenance personnel. The impact of labor 
costs is amplified because air transportation, as a service industry, is heavily dependant 
on the availability of qualified labor in almost all aspects of its operations. Thus, labor 
typically accounts for 35% of an airline's operating expenses, and threats of a strike often 
force airline operators to consent to union demands, whether the revenue structure can 
support them or not [Ref. 24 and 26]. Labor costs can be specially burdensome when 
dealing with delays because aircraft crews are limited on the number of continued flight 
hours they can operate, and additional ground personnel is required to reroute and mollify 
vexed passengers. 

Fuel and maintenance costs are the next highest for an airline, and are the most 
influenced by the type of aircraft flown. Airlines tend to prefer aircraft of the same 
family or at least of the same make to maximize part commonality and minimize 
mechanic training required. Reliability and maintenance schedules are related to aircraft 
selection and the age of the fleet. Fuel consumption also depends on engine and aircraft 
selection, but it is heavily impacted by delays due to the additional fuel burnt during 
airborne holding at inefficient cruise altitudes. 

Another cost that has become increasingly significant in airline operations has been 
landing fees and terminal space rental. Airports have been forced to expand and remodel 
to meet increased demand with the subsequent need for higher revenues, and therefore, 
higher airport use charges. While these costs account for a relatively small p>ercentage of 
the airline costs, 5% compared to 16% for promotions and sales, they have risen steadily 
increasing by nearly 70% since 1992 [Ref. 24]. 

Airport Operations 

Airports provide a vital link between the airline and its customer, and will continue to be 
an essential part of the air transportation puzzle until concepts such as personal air 
vehicles become more than mere wishful thinking. Airports also represent a significant 
source of income and a driving force behind economic growth in the communities 
surrounding them. Airports significantly contribute to pollution and noise in the 
community as well. As such, they are often driven by public interest and politics, rather 
than by their customer, the airlines. Thus, commercial airports in the United States are 
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still publicly owned, and therefore eligible for government grants and tax exempt 
financing. However, with the recent emphasis on concessions and efficient management, 
many airports have become self-sustaining in terms of day-to-day operations, and only 
require outside investment to finance airport improvement projects. 

Airport improvement projects are financed in a number of ways, all of them in some way 
resulting from the air transportation business itself. Airport Improvement Grants are 
awarded by the FAA for specific airport projects meant to improve safety, reduce noise 
impact on the community or increase capacity. These grants are funded by the Aviation 
Trust fund that collects taxes on every ticket issued in the Unites States, as well as on 
aircraft fuel. General Airport Revenue Bonds are also often used as a means to raise 
funds. These bonds are backed by airport revenue which is generated by landing fees, 
concessions, parking and rental fees, and recently also by PFC (Passenger Facility 
Charges). PFC's are approved by the FAA for a specific capital improvement project, but 
are charged by the airline on a per ticket/per segment basis, and transferred directly to the 
airport. Airline fees, which include landing fees, rental spaces and, in some cases, 
refueling and gate parking fees, account for nearly of typical airport revenues, while 
rental fees and, in some cases, a percent of gross receipts from concessions, parking, 
etc... account for the rest [Ref. 27]. 

Landing fees are typically calculated based on landing weight, but there are several cost 
calculation procedures for overall air carrier fees. These computation methods are 
generally divided into residual and compensatory. In compensatory concepts the airport 
assumes the risk and benefit of concessions, and airlines are charged based solely on their 
use. This type of scheme encourages entrepreneurship in airport management in order to 
generate enough revenue from other sources to cover costs not offset by airline fees. 
Compensatory methods can be further divided into standard and commercial 
compensatory. Standard arrangements result in both terminal rental and landing fees for 
the airline based on their use. The commercial compensatory approach distributes all 
costs among all users through rental fees only. Residual rate making methods are 
generally found at airports with less mature revenue streams since they allow the airport 
to share operating risk with the air carriers. Under these strategies airlines supplement 
other airport sources of revenue to safeguard airport profitability. Thus, well managed 
concessions result in lower airline fees, but the airline must assume any costs not offset 
by other revenue sources. This shared risk gives the airlines more leverage on capital 
investment decisions, but may harm airline competitiveness, specially on connecting 
flights, if revenue from other sources falls increasing the airline burden. Residual fee 
calculation methods can be further divided into cost center vs. airport system residual. 
This classification is based on whether all airport system expenses are tallied against 
system-wide revenues, or revenues at each cost center (terminal, airfield, etc...) are 
compared to expenditures at that center only in order to calculate the remaining costs that 
must be covered by the airline [Ref. 28]. 

Air carrier fees are the most significant airport influence on airline profitability, however, 
there are other airport characteristics that can also impact airline operations. Inefficient 
runway and taxiway layouts can significantly increase taxi times and fuel burnt. These 
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factors can be further increased by congestion and lack of peak capacity at the airport or 
its surrounding airspace. Nighttime capacity at airports is often limited by community 
concerns and some airports have curfews that eliminate night traffic altogether. Airlines 
that operate a minimum cost point-to point service, like Southwest, rely on very short 
turn-around times to maintain profitability, and can be severely affected by inefficient 
ground handling. Hub-and-spoke airlines are also affected by inefficient ground handling 
and terminal layouts, especially in terms of customer convenience and quality of service. 
Thus, airports are not all alike, and while they generally have a de facto monopoly as far 
as local air traffic is concerned, there is competition for connecting passengers, and 
therefore, for the establishment of airline hubs where expansion potential becomes an 
important factor [Ref. 29]. 

Air Traffic Control 

Air Traffic Control facilities are another factor that can differentiate airports from one 
another. TRACON (Terminal Radar Approach CONtrol) centers, for example, control 
aircraft during the climb and descent phases, but certain centers control several major 
airports as in the New York metro area, whereas others are concerned mainly with one 
airport, as is the case at the Atlanta center. Thus, there are 236 TRACON centers vs. 450 
major airports. Beyond the vicinity of major airports, 22 en-route centers guide traffic 
through the airspace sectors during the cruise segment of flight [Ref. 24 and 30] 

Given the current infrastructure a typical flight begins with the filing of a flight plan to 
inform ATC of the intended flight path, the amount of fuel on board, and alternate 
airports reachable in case of an emergency. After obtaining approval for the plan, as filed 
or with pertinent modifications, the aircraft pushes back from the gate when it is given 
permission to do so by ground control. Ground control also guides the aircraft through 
the taxiways to the runway of departure coordinating with other ground traffic. Control 
is then handed over to the tower which supervises the crew through the entire takeoff 
procedure and dictates the initial heading to follow thereafter. At that point the 
TRACON center in charge of that area handles the aircraft until it reaches cruise altitude 
and speed where it is handed off to the controllers at the appropriate en-route center. The 
sequence is reversed as the aircraft approaches its destination. Thus, the flight was under 
constant supervision for the entire mission, with ATC assuring proper separation with 
other traffic. When one considers the number of aircraft in the air following this 
procedure, as well as a number of other aircraft flying under VFR (Visual Flight Rules) 
and maintaining their own separation in good weather, but still tracked through the 
airport airspace, it is no wonder that ATC is approaching saturation [Ref. 24]. 

The capacity of the existing system can be improved by addressing system inefficiencies 
and limitations, thereby facilitating maximum use of runways and airspace sectors. 
Currently a number of safety buffers both laterally and longitudinally are built into every 
runway operation and these buffers are further expanded in poor weather conditions. 
While safety is the paramount objective of ATC, these buffers are often far larger than 
they need to be in order to avoid wake vorticity or aircraft collisions. This is due to 
inaccuracies in positioning measurements, communication delays and uncertainties in 
vortex propagation. While addressing these inefficiencies will increase the number of 
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operations a given runway can handle, this is not the only source of congestion. Today's 
Air Traffic Control system depends on the ability of its controllers to sequence, direct and 
track traffic. In heavily congested sectors controller workload becomes a significant 
constraint and traffic must be diverted or rerouted to avoid compromising safety. The 
issue of controller productivity is addressed by technologies which aim to provide 
decision support tools for the controller, as well as allowing pilots to maintain their own 
separation through airborne hazard avoidance systems. Some of the more innovative 
approaches to ATC capacity issues consider the use of GPS (Global Positioning System) 
and satellite based systems to locate and guide aircraft through all phases of flight. Free 
flight environments have also been suggested where collision avoidance is the 
responsibility of the pilots, rather than a centralized control entity, thus eliminating the 
ATC bottleneck, and alleviating congestion. 

Regardless of the technology options considered, ATC, airport, or aircraft related; a 
means to translate this qualitative discussion of the NAS components into a quantitative 
NAS model is vital to the validity of a technology assessment. 

Survey of Existing Models 

The implementation of a methodology to assess capacity technologies hinges on the 
availability of a suitable model of the system to be analyzed. Such a model would have 
to capture all aspects of the NAS relevant to technology implementation, which include 
not only each of the components (aircraft, airline, airport and ATC), but also the 
additional pressures mentioned earlier (economics, environment and safety), with special 
emphasis on the interactions between all of these ingredients. Since certain portions of 
the NAS have been previously investigated, the first step towards creating a 
comprehensive model is to obtain those component models that are currently available 
and, for those areas that have seen multiple modeling efforts, to select the models best 
suited to the task of technology evaluation. 

Aircraft 

For the purposes of a NAS model, aircraft are generally characterized in very little detail. 
The factors generally considered are approach and departure speed, cruise altitude and 
speed, ROT'S, number of passengers and fuel consumption data. However, a number of 
the technologies to be implemented require additional equipment to be installed in the 
aircraft affecting its weight and cost. Furthermore, takeoff and landing profiles are often 
quite complex requiring additional information about climb rates, and new aircraft 
concepts may find their justification in the effects they have on system congestion. 
Therefore, a more complete representation of the aircraft is desired within this NAS 
modeling effort. 

FLOPS (FLight Optimization System) is the model of choice for the definition of fixed 
wing aircraft. This synthesis and sizing code originally developed by NASA has been 
extensively modified at the Aerospace Systems Design Laboratory (ASDL) to expand its 
capabilities. Currently FLOPS is capable of scaling aircraft configurations, in terms of 
geometry, weights, and propulsion requirements, to meet a specified mission. This 
capability is necessary to model aircraft not currently in existence such as the A380. 
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Beyond this basic sizing capability FLOPS also includes a detailed takeoff and landing 
module which includes all current FAA safety requirements. Furthermore, this model is 
also linked to a noise module capable of calculating the noise footprint area of a given 
aircraft, and it can generate the information necessary to estimate CO 2 and NOx 
emissions if the engine deck used in the analysis contains the emissions information for 
the flight conditions. This information is invaluable when considering the introduction of 
new aircraft to airports surrounded by residential areas. 

The economic impact of changes on the aircraft is accounted for through the link of 
FLOPS with ALCCA (Aircraft Life Cycle Cost Analysis). FLOPS has also been used by 
ASDL in previous technology assessment projects, and contains a number of technology 
dials referred to as Kappa Factors. These Kappa factors represent a percent increase or 
decrease in a particular performance measure. All of these capabilities coupled with the 
readily available expertise with the FLOPS code make it a candidate to model the aircraft 
portion of the NAS. 

With the introduction of innovative transportation concepts such as SATS and personal 
air vehicles, other synthesis and sizing tools may be needed capable of modeling small 
aircraft, or even rotorcraft. GTPDP (Georgia Tech Preliminary Design and 
Performance), also available at ASDL, is another synthesis and sizing tool which can be 
used for smaller aircraft. VASCOMP (V/STOL Aircraft Sizing COMputer Program) is 
used to size rotorcraft, including innovative concepts such as tiltrotors. These codes may 
be included in a NAS modeling and simulation environment. However, a means to 
capture the infrastructure related to implementation of such cutting edge programs, as 
well as an understanding of their style of operations, since they will fall outside of the 
traditional airline operations model would also be necessary. 

Airline 

Airlines have multiple options when building their schedules. They can choose when and 
how to serve demand, however, their choices may affect their market share and their 
profit margin, ultimately affecting whether there is a demand to be served. The MITRE 
Corporation has developed a model of airline behavior called IMPACT (Intelligent agent- 
based Model for Policy Analysis of Collaborative TFM). In this code airlines are 
modeled as agents driven by market-share or profit depending on the airline personality 
chosen. These agents are then placed within a system with other agents representing 
ATC. A disruption such as a bad weather day is introduced in the system and the agents 
are allowed to react to the event and to each other's decisions according to their 
predefined personalities. Unfortunately, the reaction of the ATC seems to be limited to 
the activation of the Ground Delay Program which does not allow aircraft to depart if 
their destination airport is congested to avoid airborne delays [Ref. 31], The MITRE 
Corporation has also developed a model named ACSEM (Air Carrier Service Evolution 
Model) that models airline behavior in more detail. Within this model economic 
conditions, airport capacities, demand and costs are translated into a flight schedule, 
RPM, load factors, and passengers serviced along with average delays. Within this 
model transfer flights can be used to serve two cities not directly connected and 
passengers can have either a time or a cost priority. Passenger sets with similar 
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destination and time preferences are grouped together and matched to potential flights 
which are purchased based on a balance between the closeness with which they match 
customer desires, and the cost per ticket. The airline agents within the model have the 
ability to make changes to their strategies, such as varying fares and schedules, the size of 
the aircraft flown or the number of aircraft owned. As the airlines make changes, the 
flights in the schedule are flown, delays are calculated and translated into costs, and these 
costs are then balanced with the profits made. As long as the profit (or the market share) 
increases, airlines will continue to make the same type of decisions [Ref. 32]. 

Major airlines also have their own tools to determine schedules and fares, these tools are 
based specifically on the airports they operate out of and the type of fleet and strategy 
they operate under. Unfortunately, these tools are proprietary in nature, and very airline 
specific making the modeling of airline behavior and motivation beyond drives such as 
those captured in the MITRE model very difficult indeed. 

Airport 

A number of airport models exist at different levels of detail, ranging form real-time 
simulations, to quasi-analytical models, see Table 14. For the purpose of safety 
assessments the real time models are more appropriate, especially when trying to account 
for the influence of human factors. However, such tools make the simulation of the 
entire airspace cumbersome, and would not match the level of detail with which other 
portions of the NAS are being modeled. Therefore, macroscopic models are preferable in 
this case, especially those that are analytical in nature and are denoted in Table 14 by an 
asterisk. 



Scope of Model 

Level of Detail 
(type of study) 

Aprons and 
faxitvays 

Runways and final 
approaches 

Terminal area 
airspace 

En route 
airspace 

Macroscopic 
(Policy 4uial>'sis, 
oost-bcncfil studies) 


LMT Runway 
Capacity Model* 
FAA Airfield 
Capacity Model* 
DELAYS* 

AND* 


ASFM 

SDAT* 

DORATASK 

Mesoscopic 
(1‘raflTic flow 
analysis, cost- 
benciit analysis) 


NASPAC 

TMAC 

R.OWSTM 

ASCENT 

Microscopic 

(Detailed analysis 
and preliminary 
design) 

TAAM 

SIMMOD 

Same 

The Airport Machine 
HERMES 

RAMS 


Table 14: Summary of Capacity and Delay models [Ref. 33] 


The FAA airfield capacity model computes capacity for 14 common runway 
configurations. However, the situation at each airport is as different as their prevailing 
winds and surrounding landscape, both of which can affect approach paths and therefore 
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capacity. The LMI models, which are similar in structure, but differ from airport to 
airport in terms of noise restrictions and runway combinations, is thus appropriate, 
especially when combined with the queuing engine of the LMI delay model. The LMI 
delay models are also airport specific, selecting the typical configuration for each airport 
given the input weather conditions. DELAYS is also based on a queuing engine that 
treats aircraft as customers to a system with capacity dictated by the airports runway 
layout. Unfortunately, DELAYS does not consider any differences in the aircraft types 
using the airport and therefore could not accommodate new aircraft-related technology 
concepts. 

The models mentioned so far calculate the delays generated at particular airports, but the 
system-wide capacity problems are also important since many of the technologies 
proposed could affect the entire system, and it is this type of generalized effect that raises 
public interest and government funding. AND is an extension to DELAYS which 
extends this model to a network of airports, however, it has the same limitations as 
DELAYS. DP AT developed by the MITRE corporation models the NAS as a sequence 
of resources that an aircraft uses to move from its origin to its destination. Thus, it 
models capacity throughout the system, congestion arises as resources become 
unavailable, and delays when aircraft exceed the resources assigned along their route. It 
also has the ability to capture random delays which can be used to simulate weather 
events. Unfortunately, this model relies on detailed data regarding city pairs and the 
routes used to reach them. LMINet includes a net of 64 airports and the en-route sectors 
between them and has the ability to calculate cumulative delays accounting for the routes 
generally flown in current airline schedules. However, this net does not account for the 
interdependency of flights, and the ripple effects a delay early in the morning can have on 
that day's schedule [Ref. 34]. Such an effect could be accounted for through a multiplier 
that considered the time of day and length of a particular delay instance, and multiplied it 
to account for its downstream effects. 

Air Traffic Control 

A number of the technologies being proposed for the improvement of airport capacity are 
related to improvements of the Air Traffic Control system in terms of easing controller 
workloads, improving communication between pilots and the tower, or allowing for the 
reduction of inter-arrival separations. An accurate estimate of the impacts these 
technologies signify is necessary, as they will influence both capacity and airline 
behavior. IMPACT, previously mentioned, is capable of modeling the interaction 
between airlines and ATC. However, only the Ground Hold Policy can currently be 
implemented within this model. It would be of interest to expand the agent definition for 
ATC to encompass other types of policies that may be desirable for the alleviation of 
congestion in the NAS. Some of the mesoscopic models mentioned in the previous 
section: NASPAC, FLOWSIM, TMAC and ASCENT can be applied to Traffic Flow 
management problems. As could the microscopic models mentioned, TAMM and 
SIMMOD. Unfortunately, these models are very detailed requiring extensive inputs and 
complex set ups. At this point, at a simplified level, the only capability that exists is to 
model the consequences of those policies rather than the actual interactions that take 
place in their implementation. 
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Economics 

Economics are the driver behind the capacity problem as well as behind the congestion 
alleviation projects. Economics drive demand for air traffic. When trade increases the 
need for air transportation follows. When passengers have higher incomes they will often 
choose the convenience of flying over other modes of transportation. But this is only true 
as long as air transportation is affordable and convenient as delays readily translate into 
cost. Just like economic welfare drives demand for air transportation, a sharp decrease in 
air travel demand would result in fewer aircraft and associated services being required. 
This could be severe enough to have a negative impact on the economy, thus closing the 
supply and demand loop. The ACIM (Air Carrier Investment Model), developed by 
LMI, and included in their ASAC (Aviation Systems Analysis Capability) suite, aims to 
capture this two way relationship between demand and economics and is also capable of 
translating demand into the number of aircraft required to serve it, while accounting for 
retirement of aircraft currently within the fleet. The airline operating margins are 
calculated as the sum of a series of cost drivers multiplied by a productivity factor. 

The econometric based demand calculated by ACIM could be useful, however, ALCCA, 
originally developed at NASA, and subsequently improved at ASDL, considers both 
aircraft and airline costs in more detail. Furthermore, it includes a number of features 
that make it well suited for this task. Specifically, the airline costs account for the 
indirect effects of delays and lack of aircraft availability through the revenue loss module 
added in-house. However, this revenue loss module is currently based on some basic 
assumptions about airline schedules and could benefit from actual estimates of delays 
[Ref. 35] 

Unfortunately, these economic models do not capture a significant part of the 
implementation costs of some of the technologies under consideration. Namely they do 
not include costs incurred by airports and ATC during expansion and upgrade projects. 
Furthermore, the aircraft acquisition cost model may not be detailed enough to capture 
the actual cost of navigation aids and display technologies under consideration. 

Environment 

The analysis of government budgets and motivations is much too complex to include in 
this modeling effort. However, the tangible effects of government policies such as noise 
and emissions regulations could be modeled. The aircraft noise module within FLOPS 
can be used to assess whether new aircraft designs will meet the established regulations, 
and the noise module within ASAC, based on the FAA's INM (Integrated Noise Model), 
can then be used to calculate aggregate airport noise footprints and analyze how the 
community noise restrictions affect approach and departure paths. The effect of these 
paths on airport efficiency can be very significant and cannot be overlooked if an 
accurate estimate of system capacity and average delays is to be obtained. 

FLOPS engine related data can also be used to estimate CO 2 emissions, and in some 
cases also NOx generated. It would be of interest to track these in order to address 
community concerns with increased airport use. 
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Safety 

The NASA approach to safety is based on a three-prong approach. The first step in the 
approach is the modeling and simulation of past accidents to identify their causes. Once 
the causes for accidents are understood an effort can be made to prevent those accidents 
using the same calibrated modeling and simulation environments. The third step in the 
NASA approach is accident mitigation and crashworthiness [Ref. 36]. This approach 
translates to that modeled by a number of safety assessment codes: trajectory generation, 
trajectory simulation, and conflict resolution. A number of conflict avoidance models 
along with a summary of their modeling capabilities is shown in Table 15. 

Table 15; Summary of Conflict Resolution Models [Ref. 33] 


Model 

Trajectory 

Generation 

Trajectory 

Simulation 

Conflict 

Resolution 

Multi-aircraft 

Capability 

ARC2000 

Input 

3D 

Rule-based 

Pairwise 

ASIM 

Auto 

Node 

None 

None 

BDT 

Input 

3D 

Algorithmic 

Complex 

FLOWSIM 

Input 

Node 

Delay 

None 

NARSIM 

Input 

3D 

Human 

Human 

RAMS 

Auto 

3D 

Rule-based 

Pairwise 

SIMMOD 

Input 

Node 

Delay 

None 

TAAM 

Input 

3D 

Rule-based 

Pairwise 

TMAC 

Input 

3D 

None 

None 


All of these models revolve around highly detailed descriptions of the aircraft trajectories 
as well as the dynamic behavior of the aircraft involved. This level of detail is difficult to 
capture at the NAS level. 

Safety seems to be an elusive concept to model, with a particularly difficult balance in the 
detail captured vs. the complexity of the code. This is further complicated by the human 
error factor in most accidents, as most models that can include human behavior require 
in-depth detail, and real-time simulations. The most reasonable approach in this case 
may be to consider the various scenarios that can occur, attach a probability of 
occurrence to each possibility, and then estimate what the effects would be in each 
situation. For example, if one were to reduce separation between incoming aircraft, at the 
simplest level we could have a 'nothing happens' scenario, a 'recoverable vortex 
disturbance', and a 'fatal vortex disturbance'. But even this approach is as complex as the 
safety problem itself. 

One of the biggest issues in safety modeling at the NAS level is the disparity in the level 
of detail required to identify technology effects, as compared to the high level metrics 
used to define safety goals: accident and incident rates. Even if the resources were 
available to run a detailed safety model relating to each technology to be considered, a 
means to relate those safety estimates back to the NAS level would be necessary. This 
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disconnect is illustrated in Figure 30, and would require a detailed decomposition of the 
safety problem to a point that technology impacts can be directly estimated, as well as a 
means to roll those technological impacts up to the metrics that are often used for setting 
future safety goals. 



SMA, AILS, 
AVOSS, etc . . . 


♦ 


^Technologies 


Figure 30: The Safety Disconnect 


Integration 

The intention of this task is to obtain an estimate of technological effects throughout the 
NAS, rather than in a particular research area. Therefore, it is the integration of the 
models chosen that becomes the central piece of the problem. 

The NAS is a complex system whose components are intricately related to each other. 
For example, economics influence demand; excess demand and insufficient capacity lead 
to congestion that causes delays. Delays increase airline costs and fares which in turn 
also influence demand. Furthermore, a sharp drop in demand could cause a drop in 
employment within the air transportation sector influencing economics. Airlines 
represent the market for aircraft manufacturer's, therefore airline policies, which are in 
turn affected by ATC regulations, influence what type of aircraft are produced and may 
even influence the cost of the aircraft. Noise and emissions regulations can affect airport 
capacity, and may influence aircraft designs as well. Safety is affected by congestion, 
airline policies, ATC regulations, the types of aircraft flying and many other factors. 
These are just a few of the interactions present in the system. A number of methods may 
be chosen to actually implement the links discussed, and a balance must be attained 
between the amount of information captured, and the complexity of the system created. 
This is particularly important when the intent is to carry out a statistically based analysis 
since a large number of code executions may be required. 

The codes to be linked in a NAS simulation environment are varied in nature, analytical 
codes in different programming languages, agent-based models, real-time models, 
knowledge based systems... the list goes on. They may also run on different platforms 
and machines. Thus, the selection of a linking method is not an easy one. Direct linking 
of codes is an option between codes of the same language. RSM can be used to represent 
complex codes provided internal constraints can be modeled individually. The linkage 
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between codes of differing natures could also be implemented through an integration 
environment such as IMAGE (Integration Modeling and Analysis Graphical 
Environment) [Ref. 37], developed at ASDL, or a commercially available toolbox such as 
iSIGHT or Model Center. The Logistics Management Institute uses such an approach in 
their Executive Assistant, to link a number of their tools. Model Center is perhaps the 
most versatile option for its cross-platform capability. Wrappers developed for each code 
selected can be published through their Analysis Server and integrated within the Model 
Center® environment [Ref. 38]. 

Models Selected 

The most suitable models for each component of the NAS were selected from the model 
survey. And a preliminary attempt to integrate their influences on each other resulted in 
the environment depicted in Figure 31. Unfortunately, it quickly became apparent that 
within the scope of this task such a modeling and simulation environment would not be 
feasible. A number of areas required development of new models, and differences in 
detail levels arose when attempting to combine safety and ATC models with the 

□ Available 



Added Equipment 


macroscopic models available form the Logistics Management Institute. 

Figure 31: Preliminary NAS Simulation Environment 

Therefore, the problem was scoped down, focusing on the affordability aspect of 
technology implementation. This aspect was chosen for its importance both in 
motivation for the task, as well as for its significance in objectively modeling both 
benefits and drawbacks of technology programs. The scoped down NAS modeling 
environment is displayed in Figure 32. The models selected to complete this task are 
described in detail below. 
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Figure 32: NAS Modeling - Affordability Focus 


Aircraft Modeling 

The NAS modeling was, in this first approach to the problem, limited to commercial 
fixed wing aircraft. Modeling of SATS and rotorcraft concepts would require further 
development in the areas of infrastructure costs and capacity estimates and were deemed 
beyond the scope of the task. 

In order to capture aircraft related capacity improvement concepts FLOPS was included 
as part of this integrated environment. Some of the qualities that made FLOPS ideal for 
this modeling effort were already described previously. An additional advantage to this 
code is that sample files are available from previous studies. These sample files include 
inputs for a number of different aircraft including the entire range from 50 to 600 
passengers. For example, the 600 passenger data was used as a case study for Task 3 in 
this contract. 

FLOPS inputs are in text format and grouped under the namelist format. The main inputs 
required include a mission definition in terms of range, payload etc..., engine data which 
can allow the program to set its own engine parameters, or can take an input engine deck, 
and configuration data regarding wing type etc. . . 

A FLOPS input file may also include ALCOA related outputs if that economic analysis 
option is selected. 
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Airport Capacity and Delay 

The models available from the Logistic Management Institute for capacity and Delay 
were chosen for this study, specifically those relating to the Atlanta Hartsfield 
International Airport (ATL). As discussed in subtask 1, an error had been found in the 
use of runway occupancy times and a new model for a single runway was employed for 
that task instead. In combining this model to calculate capacities for the typical ATL 
runway configurations a number of additional algorithms were supplied by LMI to 
calculate arrival and departures in closely spaced parallel runways. LMI also undertook a 
revision of the typical configurations to be considered in order to follow those described 
in the FAA capacity benchmark [Ref. 39]. 

The first step in using the capacity LMI models is to provide information about the 
aircraft mix utilizing the airport, the environment that the airport is subject to, and the 
separation matrices dictated by ATC. An initial calculation is done to find single runway 
capacities based on the probability dictated aircraft sequencing, the approach, departure 
speeds and Runway Occupancy Times (ROT) of the aircraft types defined, and the 
separations that must be maintained to comply with FAA rules. When noise or other 
restrictions dictate longer approach or departure paths this is also taken into account. The 
single runway occupancies are subsequently combined according to the airport runway 
configurations, not simply in an additive manner. The configurations that yield the 
highest capacity are then used to generate airport Pareto frontiers for each weather 
condition. For the ATL version of the LMI capacity model, the task of combining 
runways and selecting the correct configuration based on weather conditions is actually 
part of the delay model. 

The LMI delay models generally run through a number of days, selecting the Pareto 
frontiers to define capacity according to weather conditions, and comparing that capacity 
to the forecasted demand for that day. The weather data is taken in hourly increments 
from a typical weather year. The demand is generated by taking the current airline 
schedules and incrementing them according to an input percentage increase in demand 
while accounting for the average number of seats per flight [Ref. 40]. 

The LMI models were modified to allow for the inclusion of the planned fifth runway at 
the Atlanta airport, as well as to track both capacity constraints MIT and ROT. In 
addition interim data regarding time of day and length of delays was output, as well as 
the standard averages generated by the typical LMI model outputs. 

Estimating Demand 

Demand estimates can be based on econometric calculations such as those found within 
the ACIM demand generation module, based on GDP unemployment, per capita income, 
etc... However, this will only generate an estimate of demand due to local economic 
prosperity, and will not account for demand generated by connecting flights at hub 
airports. These estimates of demand also will not be given on a per hour basis as is 
required by the delay module, so the approach taken by LMI to take a current schedule 
and augment it according to future demand estimates is valid. For the Atlanta airport, 
demand data is available from the 2000 OAG guide on a per hour, per aircraft type basis. 
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This demand data includes an estimate of what part of that demand is generated by Delta 
airlines hub operations which is particularly useful in modeling the behavior of a 
particular airline. 

The Cost of Delays 

The cost of delays can be divided into two categories, direct and indirect costs. Direct 
costs are caused by extra fuel burnt and longer crew hours. Indirect costs are related to 
ground personnel, gate fees, and the preservation of customer satisfaction. Passengers 
generally choose air travel over other forms of transportation for its time savings, as 
delays increase and the time to get on and off a flight increases the time saved decreases 
and the value of flying diminishes. Figure 33 shows an estimate of the costs related to 
delays as a function of length of delay, and includes comments about the number of 
passengers that will be dissatisfied enough to choose another airline or another mode of 
transportation in their next trip. 



ticket on this airline 


Figure 33: Cost vs. Delay Time [Ref. 41] 

ALCCA has the ability to capture both the direct and indirect costs of delays thanks to a 
module added at ASDL to capture revenue-losses related to delays. This module was 
separated from ALCCA for this task in order to enable it to take actual data regarding 
time of day and length of delays directly from the LMI models, rather than using the 
internal assumptions required when only ALCCA data is available. 
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Airline Economics 

Airline economics are a very complex process. As mentioned earlier, ticket pricing and 
airline marketing are more of an art than a science. However, ALCCA can calculate the 
costs incurred by airlines in their daily operations, as well as their predicted cashflows 
and the resulting retum-on-investment. Unfortunately, this can only be done at the 
aircraft type level, and successive runs are required for each representative aircraft type. 
Much of the data required to as an ALCCA input is generated by FLOPS which is also 
run for representative aircraft types. 

ALCCA was modified to explicitly include taxi times in its operations costs since many 
of the delays are incurred while at the tarmac waiting to take off (see Figure 34) and 
certain technologies such as SMA aim to reduce taxi times and increase safety in ground 
movements. 



Figure 34: Aircraft Lined up for Takeoff [Ref. 42] 


Models Developed 

Modeling efforts have been under way for various components of the NAS for years. 
Codes resulting from such modeling efforts were identified and selected for this 
comprehensive simulation effort. However, certain areas were still lacking suitable 
models, or the models existing in these areas could not be obtained. Therefore, certain 
simplified models were developed. It should be noted that, were other more detailed 
codes to become available, the codes described herein should be easy to replace while 
maintaining the same types of interactions thanks to the modularity obtained by using the 
Model Center toolbox for their integration. 

Airline Decision Making 

Airlines dictate how passenger demand is served, and hence what kind of traffic each 
airport sees, as well as the distribution of that traffic in terms of fleet mix and times of 
day. These inputs to the LMI models can be varied and their effect on capacity can be 
assessed. However, if the motivation behind those policies is not captured somehow, the 
feedback of those capacity changes to the airline revenues will not be represented. For 
example, an airline could chose to serve demand with two smaller aircraft, or one large 
jet. The large jet will result in lower costs per passenger, and have a positive effect on 
the congestion problem, but the airline may choose to use two smaller aircraft because of 
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the marketing potential. Such a decision could not be captured if the airline decision 
making process is not modeled. 

Airline decisions are based on a delicate balance of profit and market share, their product 
is the schedule they offer which involves selecting aircraft types and flight times. The 
fleet mix feeds into the LMI capacity model, the flight times feed into the delay model. 
This product is driven by demand, and costs. The costs, including the cost of delays, are 
captured by ALCCA. Demand can be based on customer satisfaction partly dictated by 
the curve shown in Figure 33. 

A simple knowledge-based system representing market share as a function of number of 
flights available during the day, the ticket price and the delays incurred was created. This 
knowledge-based system also generates recommended actions given the calculated 
market share and ROI. Thus, as a change in any of the components of the NAS is 
implemented its effect on airline policy, and the likely airline actions taken by the airline 
are captured. For example, slot pricing of peak arrival times could force the airline to use 
larger aircraft to serve its demand, but that may increase the separation required between 
incoming aircraft, reducing the overall capacity of the airport and resulting in delays, 
whether those delays are more or less than before the slot pricing is implemented will 
determine whether the airline continues that course of action or not. This is patterned in 
part after the ACSEM model which was not accessible for this task, however, the model 
generated is not agent-based in nature, and the iterative process is not automatic. The 
model only generates recommended actions; it does not execute them. 

Infrastructure Costs 

The influence of landing and other airport fees on airline profitability has been increasing 
in recent years, therefore, an estimate of how new infrastructure proposals will affect the 
behavior of airlines is needed. In order to do this a model representative of airline 
behavior must be combined with reasonable estimates of the costs related to technology 
implementation. 

The costs of technology implementation in terms of new navigation aids and new runway 
layouts can affect airline costs in two ways. By increasing the landing fees the airline is 
required to pay, and by increasing the costs of aircraft equipment. An attempt was made 
to capture the cost of new navigation equipment to be installed in aircraft. Unfortunately, 
the data available in this area was either at a high level of aggregation, such as the total 
avionics costs defined in ALCCA, or in very detailed terms such as the specific part 
number that would be required from a Rockwell Collins parts catalog. Neither level 
seemed very adequate, the technologies proposed are not described in terms of part 
numbers, and the total avionics cost is a very coarse measure since only certain portions 
of the avionics would be affected by each technology. As such, technologies relating to 
avionics equipment are only captured by an increase in avionics cost and could not be 
modeled in detail. 

The other aspect of airport and ATC improvements is reflected in infrastructure 
investments such as new runways. Such costs are propagated to the airlines using those 
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airports through increased costs ranging from landing and rental fees to ticket taxes and 
PFC’s. Landing fees are included in the costs estimated by ALCCA and can be increased 
as appropriate. PFC’s are approved by the FAA and applied across the board to all 
airlines using the airport, so their effect is equal for all airlines and will not affect their 
competitiveness, with the exception of connecting flights where an airline with a different 
airport hub may have an advantage. 

The first step in estimating airline costs due to airport improvements is to obtain a 
reasonable estimate of the actual costs that need to be covered by the increase in fees. 
This is not an easy task given the number of factors that determine runway construction 
costs, from the location of the airport to the type of landing aids incorporated. Estimates 
for the cost of constructing Atlanta’s fifth runway were obtained from the Environmental 
Impact Statement (EIS) required by the FAA to approve the project [Ref. 43]. This 
report also contained date regarding what portion of those costs would be covered by 
Airport Improvement grants, vs. what percentage would have to be covered by an 
increase in airline fees. A more generalized estimate could be created taking data for 
several airport runways and establishing a correlation between runway characteristics and 
runway costs. Unfortunately, cost data, even for the few runways that are currently 
planned is not easily accessible. The data for ATL was available only because the library 
at Georgia Tech was selected as a repository, but the EIS report was not listed in the 
library catalog and required the help of several librarians to locate it. 

Delay Propagation 

Another aspect that is often overlooked in the estimating of delays is the interdependency 
of airline schedules in terms of equipment and crews. This type of scheduling policy 
practiced by most hub-and-spoke airlines sets up itineraries where an aircraft or its crew 
is expected to be at a certain location at a certain time to continue on to another 
destination or perform a return trip. If the first flight is delayed, then all subsequent flight 
that relied on that aircraft or crew are also delayed. Attempts have been made to capture 
such effects by modeling entire airline schedules, including the itineraries followed by 
each of the resources, aircraft and crew, throughout the day. Unfortunately, this is a very 
time consuming, complex task that requires in depth knowledge of schedule recovery 
procedures. Such a study was performed on a typical American Airlines schedule and 
delay multipliers as a function of the time of day and length of delay were calculated in 
reference 44. The delay multiplier is defined as 

Delay Multiplier = (Initial Delay + Downline Delay) / Initial Delay 

Thus, assuming another hub-and-spoke airline is going to have a similar degree of 
connectivity, the initial delay calculated by the LMI models can be tracked, including the 
time of day at which it occurred, and then multiplied by the appropriate delay multiplier 
to estimate the true delay experienced and its associated costs. 

Interactions Captured 

The goal of creating a modeling and simulation environment of the NAS was to capture 
the interactions between the various areas that are generally modeled independently. The 
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basis for this goal was to capture true effects, both positive and negative, of technology 
implementation. Hence, special attention was paid to the affordability aspects of the 
NAS. 

Thus, economics are modeled as the driving factor behind air travel demand, especially in 
terms of future demand growth. Airline operating costs, especially the costs related to 
delays and loss of customer satisfaction, are also considered as a contributing factor to 
that demand insofar as they affect ticket price. The behavior of those airlines is included 
in the model to represent the link between the passenger demand and the fleet mix and 
flight schedules required to execute the LMI capacity and delay models. Furthermore, 
economics play an essential part in the estimation of technology effects when one 
considers the cost of expanding infrastructure in terms of increased landing fees or 
increased avionics costs for new navigation equipment. 

The model developed considers economics to be the puppet master pulling the strings and 
driving passenger, airline and airport decisions. Thus, the economic relationship between 
passenger demand and airline service, in terms of market share and ticket price, is 
considered of outmost importance, as is the symbiotic love-hate relationship existing 
between airports and the airlines using them. 

Subtask 3: Technology Assessment 

A modeling and simulation environment capable of capturing the interaction between 
components of the NAS has been developed enabling a technology assessment that 
considers the effect of technology programs in a broad sense, rather than the typical 
studies focused on establishing the benefits a particular technology can foster. 

Technology Identification, Evaluation and Selection 

The TEES methodology as developed at ASDL consists of eight steps, taking the designer 
from the problem definition to the selection of the best technological alternative. These 
eight steps, shown in Figure 35 are summarized below. 


Techniques or Inputs 


Jterc- 







fj.u. 

e 


lien' 


itivt 


Results or Outputs 


Figure 35: Ties Steps [Ref. 45] 
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Define the Problem 

Generally by using management tools such as Quality Function Deployment to define 
customer requirements, and map those to system requirements. In this case the traveling 
public could be identified as the customer who desires a convenient, safe and timely air 
transportation service. This step should also generate a description of the system in terms 
of its main characteristics, as well as a means to measure system performance and 
customer satisfaction. 

System Characteristics 

The National Airspace System has been described in detail in the process of developing a 
suitable modeling and simulation environment for this task. Suffice it to say here that it 
is made up of four key players: the airline, the airplane, the airport and ATC and that 
these components are deeply intertwined with very complex interactions. It was also 
noted that economics are a pervading influence throughout the system and that 
government pressures and safety were additional influences in the rules established for 
the regulation of the system. 

The Problem: Congestion and Delay 

The cost and inconvenience of delays have caused much of the current attention the 
capacity problem has been receiving. Congestion is often blamed as the sole cause of 
those delays; however, the relationship between these two concepts is more subtle. 

Congestion occurs when demand approaches or exceeds capacity. This can be due to a 
planning or an operational problem. The planning problem reflects scheduling policies 
such as non-uniform distribution of operations and the smaller-more-frequent approach to 
serving demand. Congestion due to this type of problem is present in normal operating 
conditions and it is not transient, but will remain as long as policies are not changed. The 
second type of congestion to be considered is operational in nature, and therefore harder 
to predict and prevent. This type of congestion is due to temporarily reduced capacities, 
as in the case of adverse weather conditions or malfunctioning equipment [Ref. 46] 

Planning congestion can often only be addressed through direct methods that target the 
source of the problem, such as the number and time of flights, the seats served per 
departure, or the number of runways. Operational congestion, however, is often 
addressed through lest costly indirect methods by redistributing the excess demand to less 
congested and less expensive portions of the system. An example of such an approach is 
the FAA's Ground Delay Program which transfers airborne congestion to the ground. 
Ground congestion is preferable because of the reduced cost involved since fuel usage is 
higher in the air. Ground congestion is also preferable to airborne congestion from a 
safety standpoint since a stationary aircraft on the ground is less likely to create a safety 
hazard than one circling in the air unable to stop. Metering aircraft through a control 
point or rerouting them around a point of reduced capacity are other examples of indirect 
congestion reduction approaches. It should be noted that these approaches reduce 
congestion, and improve safety, but they may also increase delay [Ref. 47]. 
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The Ground Delay Program yields a system-wide delay minimum provided uncertainty is 
ignored, en-route capacities are infinite and the arrival airport is always the final 
destination of an aircraft. The reason this congestion alleviation procedure can result in 
additional delays is these assumptions are not truly representative of the system. Flights 
are generally not independent so that an aircraft flying to New York City may then 
proceed to Chicago before returning to its original departure airport. Although airports 
represent the main bottleneck in the US airspace, en-route capacities are in fact limited by 
the ATC equipment and controllers in the sector. The NAS and its congestion are 
definitely not deterministic. Scheduled flights do not represent a constant demand, end 
even if that were the case, travel time is never fully deterministic, influenced by many 
factors like traffic volume or weather. Capacity is also not deterministic, since it depends 
on many unknown factors such as weather, the controller on duty, and the homogeneity 
of the traffic in the sector [Ref. 48]. In fact, capacity can be considered to have both an 
stochastic and a dynamic element. Stochastic in so far as it depends on weather which has 
a stochastic nature, and dynamic in so far as weather reports become increasingly 
accurate and can better predict capacity as a given time approaches [Ref. 49]. 

Thus, delays are a consequence of congestion, but the relationship may not be entirely 
direct. Furthermore, planned congestion and its associated delays are often built into 
airline schedules. Delays are only recorded when aircraft exceed their scheduled time by 
at least 15 minutes. Therefore, to improve customer satisfaction and obtain positive 
timeliness reports, airlines inflate the scheduled arrival time to include a certain delay 
margin [Ref. 40]. Consequently delays may be a biased metric, underestimating 
congestion and making other responses, such as throughput during peak periods, better 
predictors of system performance. 

System Metrics 

The purpose of this research is to assess the impact of capacity and throughput 
technologies and select those with the most promising returns. However, in order to 
compare the performance of the various technologies within the NAS, a series of metrics 
representative of the system are needed. 

Capacity 

The most obvious metric is capacity, which is defined as the maximum number of 
operations that can be performed during a fixed time interval. Airspace capacity depends 
on many factors such as meteorological conditions, runway configurations, 
arrival/departure ratio and fleet mix. However, capacity cannot be treated as a 
deterministic value since it is also influenced by the variability in flight speed and ROT 
even within a particular aircraft type. Furthermore, capacity can also affected by airspace 
factors so that, if the surrounding airspace is heavily loaded, maximum capacity may not 
be achieved. Human factors, such as controller workload, are also a source of variability 
in the capacity attainable. [Ref. 50]. 

Demand 

Demand in excess of capacity is the other ingredient of the congestion that needs to be 
alleviated. Air travel demand is generally measured in RPM or RPK, or in the case of 
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cargo, measured in RTK. As mentioned earlier, demand is driven mainly by GDP, fares 
and schedule convenience. Fares and schedules can be affected by technology infusion 
as scheduling policies change or technology costs are passed on to the customers. 
Furthermore, demand can be severely affected by the perception of safety which could 
easily be altered with technology infusion. Therefore, demand must be tracked not only 
as a factor to costly delays, but also independently to assess the impact of technologies. 

Delay 

As mentioned earlier, capacity is more representative of system performance than the 
often quoted delays which are only a consequence of congestion. However, obtaining a 
measure of delay becomes important when considering economic concerns, since there is 
a cost associated with the extra time expenditure. In fact, what phase of flight and what 
time of day that delay is accumulated would also be of interest given the different costs 
associated with airborne, ground and gate holds and the propagation of delays throughout 
the day. 

Cost 

Required average yield per RPM is an adequate measure of economic impact insofar as it 
contains the total operating costs of the airline per trip. The total operating costs include 
costs associated with delays, which they wish to minimize, as well as any increases in 
landing fees or aircraft purchase costs that may result from technology infusion. 
However, appropriate estimates of those costs that need to be accounted are also 
necessary. 

Noise and Emissions 

Noise reduction is a major thrust by airports and the administration. The intention is to 
reduce noise impact on airport communities, therefore noise regulations can restrict 
capacity, while noise reduction technologies can open up currently unavailable approach 
paths, and therefore can be considered within the capacity enhancing technologies. Noise 
is generally measured in EPNL (Effective Perceived Noise Levels) which account both 
for the noise level and the time of exposure to it or in DNL (Day-night Noise Level). 
Noise foot prints and the number of people affected by the noise could also be tracked. 

Emissions represent another concern for the airport community, especially as air travel 
continues to grow. Emissions are generally measured in terms of CO2 and NOx lbs 
released per ASM (available seat mile) and are directly related to the amount of fuel used. 
As such the effect of aircraft operations on the air quality of the surrounding areas could 
be another factor to consider when assessing the consequences of delays. 

Safety 

Typically safety is measured in terms of fatalities or damage both of which occur only 
rarely. Safety is also often measured in terms of accident and incident rates. However, 
customer confidence, and therefore demand, is most often based on 'perceived safety' an 
entirely qualitative measure that may not have any relationship with the quantitative 
measures mentioned. 
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Other 

Other metrics, such as predictability and flexibility, have been proposed as more suitable 
to capture system performance. Predictability focuses on the variable nature of the 
system and the desire to reduce uncertainty in flight time and arrival rates. Flexibility 
attempts to capture user intent, such as an airline's desire to give priority to a particular 
flight regardless on the effect on the total delay [Ref. 51]. 

Deflne Concept Space 

This step is intended to decompose the system into its components, and identify the 
potential alternatives for each component. The result of this step can be displayed in the 
form of a Morphological Matrix, where each component of the system is displayed on the 
left hand side, with its potential alternatives listed in the same row. This Morphological 
Matrix can then be used to identify a baseline and potential technological alternatives. In 
this case the NAS is composed of the aircraft, the airline, the airport, and the air traffic 
control system. Each of these components can be further decomposed to a level where 
technological alternatives can be described. For example, the aircraft could be described 
in terms of its cruise, landing and takeoff speeds, number of passengers earned, etc... A 
sample morphological matrix can be seen in Figure 36. 



Alternatives 

Attributes 

Aircraft 

Propeller 
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Large 

Very Large 

Airline 

Hub-and-spoke 

Point-to-Point 

Charter 


Airport 

International 

Regional 

Local 

GA 

ATC 

Centralized 

Shared 

Free-flight 



Figure 36: Sample Morphological Matrix of the NAS 


Modeling and Simulation 

This is possibly one of the most difficult steps to implement for a complex system. 
Striking a balance between detail captured and ease of execution can be extremely 
challenging when the system is composed of tightly interacting components. In the case 
of the NAS, models exist for individual components, but they tend to consider each 
component in isolation, not accounting for the interactions among the pieces of the 
puzzle. Furthermore, the models available for each of these components are varied in 
nature, ranging from purely analytical, to continuous time, to knowledge based, to agent- 
based. However, a simulation environment was created in Subtask 2 to generate a what- 
if environment where future technologies could be tested. 

Investigate Design Space 

With this step a better understanding of the system is desired. Identifying the major 
drivers behind the metrics of interest, recognizing trends and determining how 
uncertainty is propagated through the system are some of the goals behind the design 
space exploration. If the model lends itself well to Response Surface Methodology, a 
design of experiments can be used to screen out those inputs that do not contribute 
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significantly to the change in the response, using the remaining values to create a 
Response Surface Equation, of the form shown in below, representative of the model 
[Ref. 4 and 52]. 

n n ff-I n 

Metric = 

/=! /=! (=i 

This RSE can then be used within a Monte Carlo simulation to estimate the metric values 
an their associated probability of occurrence. An additional advantage of this approach is 
the creation of a dynamic environment called a prediction profile within the software 
package IMP [Ref. 53]. This environment, an example of which is shown in Figure 37, 
allows the designer to assess what-if scenarios very quickly. Unfortunately, RSE's do not 
estimate metric values correctly when internal constraints, which cause metric 
discontinuities, are present. The metric values and their associated probability of 
occurrence can also be obtained by applying a Monte Carlo simulation to the model 
directly, if the speed of execution is fast enough, or by using Fast Probability Integration 
Techniques [Ref. 54]. These techniques, though less restrictive than the RSE approach, 
do not yield a prediction profile environment. 



Figure 37: Sample Prediction Profile 


Evaluate System Feasibility 

Here the metric probability distributions obtained in the previous step are compared to 
the customer dictated constraints. Thus recognizing which customer desires are not 
satisfied, or how much confidence can be attached to achieving them. This information 
can then be used to establish what metrics require technology infusion, thus narrowing 
the search for suitable technologies. 
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The customer demands in this case could be as broad as safe and efficient service from 
point A to point B, or as specific as the goals NASA has set to reduce the aircraft 
accident rate by a factor of five and double the aviation system capacity within 10 years. 
Regardless, the system is not operating as efficiently as desired and as delays increase 
customer satisfaction will continue to deteriorate. 

Identify Technologies 

Potential technologies can be identified using the Morphological Matrix previously 
defined. The impacts, in terms of both benefits and drawbacks, of these technologies 
must also be identified. A probability distribution may be attached to these impacts to 
reflect technology readiness levels, the closer a technology is to being fielded, the less 
uncertain its impacts will be. Furthermore, there may be a set of technologies that are 
incompatible with each other. Identifying these technologies early on may reduce the 
size of the problem, since those technology combinations need not be tested. 

Technologies Proposed 

The nature of the technologies proposed to improve capacity and throughput is as varied 
as the components of the NAS. However, they can be loosely grouped as follows. 

Demand Management 

These technologies focus on managing the flows within the existing system to take full 
advantage of whatever capacity is currently available. They include procedures to 
redistribute the traffic flow to avoid overloading congested areas such as the Ground 
Delay Program. The use of slots with higher prices at the peak hours of the day to make 
demand more uniform would be another example. From a more innovative point of view 
the redistribution of demand with other types of aircraft could be included within this 
category also. Large aircraft such as the projected A380 could alleviate some of the 
congestion problem by carrying more passengers per operation, however, they would not 
alleviate terminal congestion. Tilt Rotors and runway independent aircraft have also 
been proposed as an alternative to regular transports provided they could operate without 
disturbing current traffic flow and their noise production could be minimized. Taking 
this idea of serving demand without altering exiting capacity further is the SATS program 
which would not affect major airport capacity, but rather would attempt to make use of 
the thousands of General Aviation (GA) airports currently underutilized. However, this 
program would have to overcome the perceived lack of safety of GA aircraft, and the 
infrastructure issues of serving large traffic and passenger volumes at those additional 
airports. 

Capacity Enhancement 

Beyond the management of demand within current capacity constraints there are also 
efforts underway to increase capacity at major airports by enabling more operations per 
runway, and expanding the number of runways available at airports. Perhaps the first 
step to accomplish this is to reduce the lead time required to allow construction of new 
runways which can be as much as 10 years. The FA A is taking steps to expedite the 
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process, however, community pressures and availability of land may be unavoidable 
constraints at some airports. 

In addition to accelerating the airport expansion process, a number of initiatives have 
been taken to increase the number of operations possible per runway regardless of 
weather conditions, and to minimize the separation required between parallel runways to 
maintain independent operations. These initiatives are currently being investigated by 
NASA, under their TAP program, with the intention of eliminating built in separation 
buffers currently necessary to account for reaction and communication time, as well as 
positional uncertainty and wake turbulence. The TAP program is developing aids in four 
main areas to include only the separation that is strictly necessary. These areas as 
described in the program objectives are: 

• Reduced Spacing Operations: To reduce lateral and longitudinal spacing in non- 
visual conditions. AVOSS (Aircraft VOrtex Sensing System) and AILS (Airborne 
Information for Lateral Spacing) are some of the enabling technologies under 
consideration. 

• Air Traffic Management: To enhance CTAS (Center-TRACON Automation 
System), integrating it with the FMS (Flight Management System) to reduce spacing 
and position uncertainty. 

• Low Visibility Landing and Surface Operations: To expedite airport surface 
operations in adverse weather through sensor and display technologies such as 
ROTO (Roll Out and Turn Off), TNASA (Taxi Navigation And Situational 
Awareness system) or DROM (Dynamic Runway Occupancy Measurement). 

• Aircraft-ATC Systems Integration: To enable clear weather operations in 
instrument-weather conditions. Cost, safety and technology demonstrations are the 
main focus of this research area. 

With all these technologies the TAP is attempting to address inefficiencies in the arrival 
stream at an airport, currently in place to maintain safety in an uncertain environment. 
Should these technologies achieve full development they would contribute to a 
significant reduction in the uncertainty intrinsic to airport arrival operations [Ref 36]. 
Figure 38 illustrates these inefficiencies and the technologies that target them. 
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Figure 38: TAP Project Approach 


Innovative Concepts 

The concept of “Free Flight”, which is also currently being explored by NASA and the 
FAA, signifies a radical change in air traffic management policy. The goal is to eliminate 
the constraints imposed by a centralized ATC system, transferring the responsibility for 
safety and efficiency to the airplane operators. Such a leap in air traffic practices requires 
development of numerous tools to maintain a guarantee of safety, especially during the 
technology transition phase. Most of these tools are intended to provide decision support 
and task-automation for controllers and pilots. The first step toward a free-flight 
environment involves the testing of those tools within the existing ATC system with 
hopes of dividing the ATM (Air Traffic Management) tasks between pilots and 
controllers at a later date. 

Thus far, certain technologies developed by NASA under their AATT (Advanced Air 
Transportation Technologies) initiative, such as pFAST (Passive Final Approach Spacing 
Tool), SMA (Surface Movement Advisor) and TMA (Traffic Management Advisor), 
have already been field tested. These technologies will continue to mature and additional 
technologies will come online as Free Flight phase 2 develops. 

But innovative concepts are not limited to a new ATC paradigm. The decentralizing 
approach to congestion relief could be taken to the extreme moving away from the 
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concept of airport-to-airport travel entirely. NASA's mobility goal is aimed at reducing 
doorstep-to-destination travel time, but whenever a change of transportation modes is 
required waiting time is inevitable. Thus, Personal Air Vehicle concepts could be 
considered as a long-term solution to lengthy travel times and terminal area congestion. 
Though congestion in the airways may then be inevitable, at least airways can be laid in 
three dimensions, whereas the highway system is limited to two. 

Evaluate Technologies 

If appropriate factors exist within the model to capture technology discontinuities a 
procedure similar to the one used to investigate the design space can be used to estimate 
the effect of technologies and their combinations yielding results in terms of the metric 
values that can be obtained with a particular confidence level. 

Select Technologies 

The results from the previous step can now be used to select the most promising 
technologies in terms of their effects on the system, and the resources required to 
implement them. 

Case Study 

The modeling environment developed and the methodology proposed can now be tested 
on a sample case. This case study utilizes technologies applied in different components 
of the NAS to demonstrate the new capability of capturing aircraft and airspace 
technologies simultaneously. It also focuses attention on the affordability issues related 
to technology implementation. 

Atlanta Hartsfield International Airport 

The Atlanta International airport was chosen as a good site for this case study given its 
large operation volume, the largest in the nation in terms of passengers in the year 2000, 
as well as projects currently underway to add a runway and continue testing of free-flight 
related technologies such as SMA. 

The Logistics Management Institute provided capacity and delay models for this airport 
with subsequent modifications to capture runway configurations not previously 
considered. These models were also modified to take advantage of the potential for 
increased capacity in the future through the addition of a third Cat III runway at the 
Atlanta airport. This fifth runway would also be available for departures thanks to its 
9,000 ft length, increased from the initially approved commuter runway. Construction of 
this runway at the Atlanta airport will be especially challenging due to space constraints. 
The Atlanta airport is set at the intersection of three interstate highways, 1-85, 1-75 and I- 
285, and the projected runway which will open in 2005 along with its servicing taxiway 
will be required to cross over 1-285 [Ref. 55]. 

Another reason this airport was chosen was the availability of its Environmental Impacts 
Statement which contains information regarding runway costs and projected demand for 
the area. 
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Terminal Area Productivity Program 

The terminal area productivity program comprises a set of air traffic control technologies 
aimed at improving the existing system, whereas the technologies developed under 
AATT are aimed at developing the decision support aids for a completely different ATC 
paradigm. The TAP technologies, especially those relating to closely-spaced parallel 
runways are specially applicable in this case since the Atlanta airport currently contains 
two pairs of runways separated by the terminal building. In normal operations, the two 
outer runways are used for independent approaches, and the two inner runways are used 
for independent departures. However, during departure or arrival pushes each pair of 
runways could be used simultaneously if the technologies to make such a proposition safe 
existed. Furthermore, the fifth runway, though nearly 4,000ft south of the existing 
runways, is still too close to enable independent operations in bad weather. In addition, 
detailed information regarding the TAP technologies is available from a previous study 
carried out at LMI and described in Reference 56 which makes validation of this 
technique possible. 

The Surface Management Advisor, though technically developed under the AATT 
program, will also be modeled in this task since it is already in testing at the Atlanta 
airport. 

Delta: A Hub-and-Spoke Airline 

In the modeling effort a simulation of airline decision-making procedures was developed 
based on the typical behavior of U. S. hub-and-spoke airlines. ATL is the main hub for 
one such airline; Delta. Some contacts were established with that airline in efforts to 
leverage their expertise in the area. Those efforts have thus far been unfruitful, but 
demand estimates for this airline specifically, as well as the airport as a whole have been 
obtained from OAG data. Additionally, Delta recently restructured its schedule to 
alleviate delay related problems. The study of such a behavior in the model, compared to 
the actual steps taken, would also be of interest. 

A380: The Aircraft of the Future? 

The goal for this task was to enable selection of technologies at the infrastructure as well 
as the aircraft level. The aircraft technology considered in this case is a new aircraft 
concept carrying far more passengers than ever before. The Boeing company abandoned 
its efforts to develop such an aircraft with the firm conviction that the tendency to serve 
demand with smaller aircraft would continue, and therefore the market for such large 
airliners would not exist. Airbus, however, continued its development of the aircraft and 
has recently released preliminary data regarding its airport compatibility [Ref. 57]. It is a 
gamble for the aircraft manufacturer, similar to that taken by Boeing in the seventies with 
the development of the B747. If the market is not there, the investment made in 
developing and manufacturing the aircraft will be lost and the company many not be able 
to withstand such a financial setback. However, if the project succeeds and the airlines 
that have ordered the craft are joined by others that have expressed interest, but no firm 
orders, Airbus will have a comer on the market. Its main market is likely to be in the still 
regulated markets of the Asia-Pacific region where land constraints make airport 
expansion prohibitive. Therefore, one of its main advantages, its ability to alleviate 
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congestion, cannot be truly captured unless an approach such as the one developed in this 
task is taken. 

Results and Conclusions 

The results of this case study will be forthcoming. Technical difficulties in obtaining the 
modified capacity and delay models from the Logistics Management Institute caused an 
unexpected delay in this portion of the task. An updated copy of that code was finally 
obtained in March of 2002 and current efforts are underway to modify that version in 
order to include the fifth runway and reintegrate it with the rest of the modeling 
environment. 
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RDT&E Cost Due To Variation in Test Programs & Mockups Complexity Factors 
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RDT&E Cost Due To Variation In Basic Factory Labor Complexity Factors 
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